Capstone-Project-aiml¶

AIML Online Capstone - Pneumonia Detection Challenge¶

Project Objectives¶

The objective of the project is,  Learn to how to do build an Object Detection Model  Use transfer learning to fine-tune a model.  Learn to set the optimizers, loss functions, epochs, learning rate, batch size, checkpointing, early stopping etc.  Read different research papers of given domain to obtain the knowledge of advanced models for the given problem.

The Real Problem¶

What is Pneumonia?¶

Pneumonia is an infection in one or both lungs. Bacteria, viruses, and fungi cause it. The infection causes inflammation in the air sacs in your lungs, which are called alveoli. Pneumonia accounts for over 15% of all deaths of children under 5 years old internationally. In 2017, 920,000 children under the age of 5 died from the disease. It requires review of a chest radiograph (CXR) by highly trained specialists and confirmation through clinical history, vital signs and laboratory exams. Pneumonia usually manifests as an area or areas of increased opacity on CXR. However, the diagnosis of pneumonia on CXR is complicated because of a number of other conditions in the lungs such as fluid overload (pulmonary edema), bleeding, volume loss (atelectasis or collapse), lung cancer, or postradiation or surgical changes. Outside of the lungs, fluid in the pleural space (pleural effusion) also appears as increased opacity on CXR. When available, comparison of CXRs of the patient taken at different time points and correlation with clinical symptoms and history are helpful in making the diagnosis. CXRs are the most commonly performed diagnostic imaging study. A number of factors such as positioning of the patient and depth of inspiration can alter the appearance of the CXR, complicating interpretation further. In addition, clinicians are faced with reading high volumes of images every shift. Pneumonia Detection Now to detection Pneumonia we need to detect Inflammation of the lungs. In this project, you’re challenged to build an algorithm to detect a visual signal for pneumonia in medical images. Specifically, your algorithm needs to automatically locate lung opacities on chest radiographs.

Business Domain Value¶

Automating Pneumonia screening in chest radiographs, providing affected area details through bounding box. Assist physicians to make better clinical decisions or even replace human judgement in certain functional areas of healthcare (eg, radiology). Guided by relevant clinical questions, powerful AI techniques can unlock clinically relevant information hidden in the massive amount of data, which in turn can assist clinical decision making.

Project Description¶

In this capstone project, the goal is to build a pneumonia detection system, to locate the position of inflammation in an image. Tissues with sparse material, such as lungs which are full of air, do not absorb the X-rays and appear black in the image. Dense tissues such as bones absorb X-rays and appear white in the image. While we are theoretically detecting “lung opacities”, there are lung opacities that are not pneumonia related. In the data, some of these are labeled “Not Normal No Lung Opacity”. This extra third class indicates that while pneumonia was determined not to be present, there was nonetheless some type of abnormality on the image and oftentimes this finding may mimic the appearance of true pneumonia. Dicom original images:

Medical images are stored in a special format called DICOM files (*.dcm). They contain a combination of header metadata as well as underlying raw image arrays for pixel data. Details about the data and dataset files are given in below link, https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/data

Pre-Processing, Data Visualization, EDA¶

 Exploring the given Data files, classes and images of different classes.  Dealing with missing values  Visualization of different classes  Analysis from the visualization of different classes.

Model Building¶

 Building a pneumonia detection model starting from basic CNN and then improving upon it.  Train the model  To deal with large training time, save the weights so that you can use them when training the model for the second time without starting from scratch.

Test the Model, Fine-tuning and Repeat¶

 Test the model and report as per evaluation metrics  Try different models  Set different hyper parameters, by trying different optimizers, loss functions, epochs, learning rate, batch size, checkpointing, early stopping etc. for these models to fine-tune them  Report evaluation metrics for these models along with your observation on how changing different hyper parameters leads to change in the final evaluation metric.

Reference¶

Acknowledgment for the datasets. https://www.kaggle.com/c/rsna-pneumonia-detection-challenge/overview/acknowledgements

Installing and importing the required libraries¶

!pip install pydicom

Collecting pydicom
  Downloading https://files.pythonhosted.org/packages/f4/15/df16546bc59bfca390cf072d473fb2c8acd4231636f64356593a63137e55/pydicom-2.1.2-py3-none-any.whl (1.9MB)
     |████████████████████████████████| 1.9MB 9.2MB/s 
Installing collected packages: pydicom
Successfully installed pydicom-2.1.2

!pip install wandb

Collecting wandb
  Downloading https://files.pythonhosted.org/packages/e0/b4/9d92953d8cddc8450c859be12e3dbdd4c7754fb8def94c28b3b351c6ee4e/wandb-0.10.32-py2.py3-none-any.whl (1.8MB)
     |████████████████████████████████| 1.8MB 8.2MB/s 
Requirement already satisfied: protobuf>=3.12.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (3.12.4)
Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (7.1.2)
Collecting sentry-sdk>=0.4.0
  Downloading https://files.pythonhosted.org/packages/1c/4a/a54b254f67d8f4052338d54ebe90126f200693440a93ef76d254d581e3ec/sentry_sdk-1.1.0-py2.py3-none-any.whl (131kB)
     |████████████████████████████████| 133kB 54.5MB/s 
Collecting configparser>=3.8.1
  Downloading https://files.pythonhosted.org/packages/fd/01/ff260a18caaf4457eb028c96eeb405c4a230ca06c8ec9c1379f813caa52e/configparser-5.0.2-py3-none-any.whl
Requirement already satisfied: PyYAML in /usr/local/lib/python3.7/dist-packages (from wandb) (3.13)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.7/dist-packages (from wandb) (2.8.1)
Requirement already satisfied: requests<3,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (2.23.0)
Requirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (5.4.8)
Collecting docker-pycreds>=0.4.0
  Downloading https://files.pythonhosted.org/packages/f5/e8/f6bd1eee09314e7e6dee49cbe2c5e22314ccdb38db16c9fc72d2fa80d054/docker_pycreds-0.4.0-py2.py3-none-any.whl
Collecting subprocess32>=3.5.3
  Downloading https://files.pythonhosted.org/packages/32/c8/564be4d12629b912ea431f1a50eb8b3b9d00f1a0b1ceff17f266be190007/subprocess32-3.5.4.tar.gz (97kB)
     |████████████████████████████████| 102kB 14.1MB/s 
Collecting GitPython>=1.0.0
  Downloading https://files.pythonhosted.org/packages/27/da/6f6224fdfc47dab57881fe20c0d1bc3122be290198ba0bf26a953a045d92/GitPython-3.1.17-py3-none-any.whl (166kB)
     |████████████████████████████████| 174kB 61.1MB/s 
Requirement already satisfied: six>=1.13.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (1.15.0)
Requirement already satisfied: promise<3,>=2.0 in /usr/local/lib/python3.7/dist-packages (from wandb) (2.3)
Collecting pathtools
  Downloading https://files.pythonhosted.org/packages/e7/7f/470d6fcdf23f9f3518f6b0b76be9df16dcc8630ad409947f8be2eb0ed13a/pathtools-0.1.2.tar.gz
Collecting shortuuid>=0.5.0
  Downloading https://files.pythonhosted.org/packages/25/a6/2ecc1daa6a304e7f1b216f0896b26156b78e7c38e1211e9b798b4716c53d/shortuuid-1.0.1-py3-none-any.whl
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from protobuf>=3.12.0->wandb) (57.0.0)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from sentry-sdk>=0.4.0->wandb) (2020.12.5)
Requirement already satisfied: urllib3>=1.10.0 in /usr/local/lib/python3.7/dist-packages (from sentry-sdk>=0.4.0->wandb) (1.24.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.0.0->wandb) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.0.0->wandb) (2.10)
Collecting gitdb<5,>=4.0.1
  Downloading https://files.pythonhosted.org/packages/ea/e8/f414d1a4f0bbc668ed441f74f44c116d9816833a48bf81d22b697090dba8/gitdb-4.0.7-py3-none-any.whl (63kB)
     |████████████████████████████████| 71kB 11.0MB/s 
Requirement already satisfied: typing-extensions>=3.7.4.0; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from GitPython>=1.0.0->wandb) (3.7.4.3)
Collecting smmap<5,>=3.0.1
  Downloading https://files.pythonhosted.org/packages/68/ee/d540eb5e5996eb81c26ceffac6ee49041d473bc5125f2aa995cf51ec1cf1/smmap-4.0.0-py2.py3-none-any.whl
Building wheels for collected packages: subprocess32, pathtools
  Building wheel for subprocess32 (setup.py) ... done
  Created wheel for subprocess32: filename=subprocess32-3.5.4-cp37-none-any.whl size=6502 sha256=166bb0e9c68825fb6190fa08487aedd4a88d6798245c7d43399a0d68cf8cb514
  Stored in directory: /root/.cache/pip/wheels/68/39/1a/5e402bdfdf004af1786c8b853fd92f8c4a04f22aad179654d1
  Building wheel for pathtools (setup.py) ... done
  Created wheel for pathtools: filename=pathtools-0.1.2-cp37-none-any.whl size=8807 sha256=54b2c0e4a6e0f2ac3abf5f729e292fa9190aa1df02bbbf163bfd7e37f0f4db6c
  Stored in directory: /root/.cache/pip/wheels/0b/04/79/c3b0c3a0266a3cb4376da31e5bfe8bba0c489246968a68e843
Successfully built subprocess32 pathtools
Installing collected packages: sentry-sdk, configparser, docker-pycreds, subprocess32, smmap, gitdb, GitPython, pathtools, shortuuid, wandb
Successfully installed GitPython-3.1.17 configparser-5.0.2 docker-pycreds-0.4.0 gitdb-4.0.7 pathtools-0.1.2 sentry-sdk-1.1.0 shortuuid-1.0.1 smmap-4.0.0 subprocess32-3.5.4 wandb-0.10.32

import tensorflow as tf
tf.__version__

'2.5.0'

!pip install tensorflow_addons

Collecting tensorflow_addons
  Downloading https://files.pythonhosted.org/packages/66/4b/e893d194e626c24b3df2253066aa418f46a432fdb68250cde14bf9bb0700/tensorflow_addons-0.13.0-cp37-cp37m-manylinux2010_x86_64.whl (679kB)
     |████████████████████████████████| 686kB 7.2MB/s 
Requirement already satisfied: typeguard>=2.7 in /usr/local/lib/python3.7/dist-packages (from tensorflow_addons) (2.7.1)
Installing collected packages: tensorflow-addons
Successfully installed tensorflow-addons-0.13.0

tf.keras.__version__

'2.5.0'

import random
random.seed(0)
import warnings
warnings.filterwarnings('ignore')

import os
import logging
import pandas as pd
import pydicom as dcm
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from matplotlib.patches import Rectangle
from skimage.transform import resize
from sklearn.model_selection import train_test_split
import tqdm.notebook as tq
from copy import deepcopy
import glob
import datetime
import cv2
import pytz
import multiprocessing
import hashlib
from sklearn.utils import shuffle
import imgaug as ia
import imgaug.augmenters as iaa
import sys, math
from imgaug.augmentables.bbs import BoundingBox, BoundingBoxesOnImage
import importlib
import wandb
from wandb.keras import WandbCallback
from keras.callbacks import LambdaCallback
import tensorflow.keras.backend as K
from tensorflow.keras.optimizers import SGD, Adam
import shutil
import pickle
from pydicom.pixel_data_handlers.util import apply_color_lut
from tensorflow_addons.optimizers import LAMB

%matplotlib inline

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
logging.getLogger('tensorflow').setLevel(logging.ERROR)

Importing the data¶

1) Mounting Google Drive¶

from google.colab import drive
drive.mount('/content/drive')

Mounted at /content/drive

2) Setting up kaggle to download datasets¶

Please setup the kaggle_path where you have kept the kaggle.json file
Or create the directory structure in your google drive like MyDrive/Colab/.kaggle/ and upload kaggle.json to this location

# Install kaggle to download the dataset
!pip install -q kaggle

# Ensure you have setup kaggle.json with the required API token and saved it to your google drive mounted above.
# Provide the location of the .kaggle/kagle.json file
kaggle_path = '/content/drive/MyDrive/Colab/.kaggle/'
os.chdir(kaggle_path)
#!export KAGGLE_CONFIG_DIR=/content/drive/MyDrive/Colab/.kaggle/

!mkdir ~/.kaggle
!cp kaggle.json ~/.kaggle/
!kaggle --version

Kaggle API 1.5.4

!pip install --upgrade --force-reinstall --no-deps kaggle

Collecting kaggle
  Downloading https://files.pythonhosted.org/packages/3a/e7/3bac01547d2ed3d308ac92a0878fbdb0ed0f3d41fb1906c319ccbba1bfbc/kaggle-1.5.12.tar.gz (58kB)
     |████████████████████████████████| 61kB 5.0MB/s 
Building wheels for collected packages: kaggle
  Building wheel for kaggle (setup.py) ... done
  Created wheel for kaggle: filename=kaggle-1.5.12-cp37-none-any.whl size=73053 sha256=fde03dcdd0bc8c547efef93b8561a1c0b78c105afdc62de2b54b45fb2f12b719
  Stored in directory: /root/.cache/pip/wheels/a1/6a/26/d30b7499ff85a4a4593377a87ecf55f7d08af42f0de9b60303
Successfully built kaggle
Installing collected packages: kaggle
  Found existing installation: kaggle 1.5.12
    Uninstalling kaggle-1.5.12:
      Successfully uninstalled kaggle-1.5.12
Successfully installed kaggle-1.5.12

# Kaggle api version should be 1.5.12 or higher
!kaggle --version

Kaggle API 1.5.12

3) Download datasets from kaggle¶

Please setup the destination path where you want to install the dataset
Or create the directory structure as MyDrive/Colab/Capstone Project/ to store the datasets in your google drive

project_path = '/content/drive/MyDrive/Colab/CapstoneProject/'
os.chdir(project_path)
base_path = project_path + 'dataset'

if not os.path.exists(base_path):
    os.makedirs(base_path)

os.chdir(base_path)
data_path = os.path.join(base_path, 'data')
lib_path = os.path.join(base_path, 'libs')
# Downloading kaggle dataset from Pneumonia detection challenge
file_path = data_path + '/rsna-pneumonia-detection-challenge.zip'
if not os.path.exists(file_path):
  !kaggle competitions download -q -c rsna-pneumonia-detection-challenge

if not os.path.exists(os.path.join(data_path, 'checkpoint', 'maskrcnn')):
    os.makedirs(os.path.join(data_path, 'checkpoint', 'maskrcnn'))

if not os.path.exists(os.path.join(data_path, 'logs', 'maskrcnn')):
    os.makedirs(os.path.join(data_path, 'logs', 'maskrcnn'))

if not os.path.exists(os.path.join(data_path, 'runs', 'maskrcnn')):
    os.makedirs(os.path.join(data_path, 'runs', 'maskrcnn'))

Dataset is ready to be imported and to perform EDA

os.getcwd()

'/content/drive/MyDrive/Colab/CapstoneProject/dataset'

import zipfile
if not os.path.exists(data_path + '/' + 'stage_2_train_images'):
  zip_ref = zipfile.ZipFile(data_path + '/rsna-pneumonia-detection-challenge.zip', 'r')
  zip_ref.extractall()
  zip_ref.close()

4) Importing required libraries for training¶

#!git clone https://github.com/akTwelve/Mask_RCNN.git
#!git clone https://github.com/WittmannF/LRFinder.git
# !git clone https://github.com/psklight/keras_one_cycle_clr.git

sys.path.append(os.path.join(lib_path, 'keras_one_cycle_clr'))
import keras_one_cycle_clr as ktool

# Importing Mask RCNN model and required libraries
sys.path.append(os.path.join(lib_path, 'Mask_RCNN'))

import mrcnn.config as config
import mrcnn.model as modellib
from mrcnn import utils
from mrcnn import visualize
from mrcnn.model import log

sys.path.append(os.path.join(lib_path, 'LR_Finder'))
import lr_finder as lr_finder

Exploratory Data Analysis¶

# function to plot PDF
def plot_pdf(data,x_label,y_label,title):
    f,ax = plt.subplots(1,1, figsize=(7.5,4))
    sns.distplot(a=data, ax=ax)
    plt.xlabel(x_label)
    plt.ylabel(y_label)
    plt.title(title)
    plt.show()

# function to plot CDF
def plot_cdf(data,x_label,y_label,title):
    f,ax = plt.subplots(1,1, figsize=(7.5,4))
    counts, bin_edges = np.histogram(data.dropna(), bins=10, 
                                     density = True)
    pdf = counts/(sum(counts))#cal pdf
    cdf = np.cumsum(pdf)#cumulative sum of pdf,calculating cdf
    plt.plot(bin_edges[1:], cdf)
    plt.ylabel(y_label)
    plt.xlabel(x_label)
    plt.title(title)
    plt.show()
# function to plot box_plot    
def box_plot(data,col,title):
    f,ax = plt.subplots(1,1, figsize=(7.5,4))
    sns.boxplot(y=col, data=data,ax=ax)
    plt.title(title)
    plt.show()

1) Duplicate data detection¶

Adding checksum while reading the images and labels

# Checksum function to detect duplicate images
def md5Checksum(file):
  with open(data_path + '/' + 'stage_2_train_images' + '/' + file + '.dcm', "rb") as f:
    bytes = f.read()
    file_hash = hashlib.md5(bytes).hexdigest()
  return file_hash

# 
labels_df = pd.read_csv(data_path + '/stage_2_train_labels.csv')
det_class_info_df = pd.DataFrame()
if not os.path.exists(data_path + '/stage_2_detailed_class_info_updated.csv'):
  det_class_info_df = pd.read_csv(data_path + '/stage_2_detailed_class_info.csv')
  # Removing duplicates
  det_class_info_df=det_class_info_df.drop_duplicates().reset_index(drop=True)
  det_class_info_df['Target'] = det_class_info_df['class'].apply(lambda x: 1 if x =='Lung Opacity' else 0)
  filelist= det_class_info_df['patientId'].to_numpy()
  # Checksum to identify duplicate images
  checksum=[]
  for file in filelist:
    checksum.append(md5Checksum(file))
  checksum = np.array(checksum)
  det_class_info_df['checksum'] = checksum
  det_class_info_df.to_csv(data_path + '/stage_2_detailed_class_info_updated.csv')
else:
  det_class_info_df = pd.read_csv(data_path + '/stage_2_detailed_class_info_updated.csv', index_col=[0])

merged_class_df = labels_df.merge(det_class_info_df[['patientId','class', 'checksum']], left_on='patientId', right_on='patientId', how='inner')

merged_class_df.head()

Checking for duplicates

print ('There are', det_class_info_df['checksum'].nunique(), 'checksum values')

There are 26684 checksum values

print('There are', det_class_info_df['patientId'].nunique(), 'patients')

There are 26684 patients

Observation: There are no duplicates images.

2) Visualizing Class distributions¶

merged_class_df[(merged_class_df['Target']==1)].groupby('patientId')['class'].count().max()

4

class_info=merged_class_df["class"].value_counts()
labels = (np.array(class_info.index))
sizes = (np.array((class_info / class_info.sum())*100))
plt.pie(sizes, labels=labels, autopct='%1.1f%%', shadow=True, startangle=90)
plt.title("Class percentage")
plt.show()

3) Visualizing targets distribution¶

fig, ax = plt.subplots(nrows=1,figsize=(6,6))
tmp = merged_class_df.groupby('Target')['class'].value_counts()
df = pd.DataFrame(data={'Exams': tmp.values}, index=tmp.index).reset_index()
sns.barplot(ax=ax,x = 'Target', y='Exams',hue='class',data=df, palette='Set1')
plt.title("Chest exams class and Target")
plt.show()

4) Analyze DICOM file construct¶

# Extract one image and process the DICOM information.
patientId = det_class_info_df.iloc[0]['patientId']
dicom_file_dataset = dcm.read_file(os.path.join(data_path, 'stage_2_train_images', patientId + '.dcm'))

# Typical dataset metadata
dicom_file_dataset

Dataset.file_meta -------------------------------
(0002, 0000) File Meta Information Group Length  UL: 202
(0002, 0001) File Meta Information Version       OB: b'\x00\x01'
(0002, 0002) Media Storage SOP Class UID         UI: Secondary Capture Image Storage
(0002, 0003) Media Storage SOP Instance UID      UI: 1.2.276.0.7230010.3.1.4.8323329.28530.1517874485.775526
(0002, 0010) Transfer Syntax UID                 UI: JPEG Baseline (Process 1)
(0002, 0012) Implementation Class UID            UI: 1.2.276.0.7230010.3.0.3.6.0
(0002, 0013) Implementation Version Name         SH: 'OFFIS_DCMTK_360'
-------------------------------------------------
(0008, 0005) Specific Character Set              CS: 'ISO_IR 100'
(0008, 0016) SOP Class UID                       UI: Secondary Capture Image Storage
(0008, 0018) SOP Instance UID                    UI: 1.2.276.0.7230010.3.1.4.8323329.28530.1517874485.775526
(0008, 0020) Study Date                          DA: '19010101'
(0008, 0030) Study Time                          TM: '000000.00'
(0008, 0050) Accession Number                    SH: ''
(0008, 0060) Modality                            CS: 'CR'
(0008, 0064) Conversion Type                     CS: 'WSD'
(0008, 0090) Referring Physician's Name          PN: ''
(0008, 103e) Series Description                  LO: 'view: PA'
(0010, 0010) Patient's Name                      PN: '0004cfab-14fd-4e49-80ba-63a80b6bddd6'
(0010, 0020) Patient ID                          LO: '0004cfab-14fd-4e49-80ba-63a80b6bddd6'
(0010, 0030) Patient's Birth Date                DA: ''
(0010, 0040) Patient's Sex                       CS: 'F'
(0010, 1010) Patient's Age                       AS: '51'
(0018, 0015) Body Part Examined                  CS: 'CHEST'
(0018, 5101) View Position                       CS: 'PA'
(0020, 000d) Study Instance UID                  UI: 1.2.276.0.7230010.3.1.2.8323329.28530.1517874485.775525
(0020, 000e) Series Instance UID                 UI: 1.2.276.0.7230010.3.1.3.8323329.28530.1517874485.775524
(0020, 0010) Study ID                            SH: ''
(0020, 0011) Series Number                       IS: "1"
(0020, 0013) Instance Number                     IS: "1"
(0020, 0020) Patient Orientation                 CS: ''
(0028, 0002) Samples per Pixel                   US: 1
(0028, 0004) Photometric Interpretation          CS: 'MONOCHROME2'
(0028, 0010) Rows                                US: 1024
(0028, 0011) Columns                             US: 1024
(0028, 0030) Pixel Spacing                       DS: [0.14300000000000002, 0.14300000000000002]
(0028, 0100) Bits Allocated                      US: 8
(0028, 0101) Bits Stored                         US: 8
(0028, 0102) High Bit                            US: 7
(0028, 0103) Pixel Representation                US: 0
(0028, 2110) Lossy Image Compression             CS: '01'
(0028, 2114) Lossy Image Compression Method      CS: 'ISO_10918_1'
(7fe0, 0010) Pixel Data                          OB: Array of 142006 elements

5) Bounding Box Analysis¶

5.1) Bounding Box X-Coordinates¶

plot_pdf(merged_class_df.x.values,'x-coordinates','Probability Density','PDF of Bounding Box X-coordinates')
plot_cdf(merged_class_df['x'],'X-coordinates','X-coordinates','CDF of Bounding Box X-coordinates')
box_plot(merged_class_df,'x','Boxplot of Bounding Box X-coordinates')

Observation:

The double-bell curve that we got in PDF of x-coordinates is due to the location of the lungs (as the opacity bounding boxes are on lungs only).
The x-coordinates passing over the lungs has high values in comparison to the area which do not have lungs.
The range that we got for x-coordinates is from 0 to 800.
Almost 99% of the x-coordinate values are less that 750.
The IQR(interquartile range) that we got is from 200 (25th percentile) & 600(75th percentile).
The median value that we got is 300(approx)

5.2) Bounding Box Y-Coordinates¶

plot_pdf(merged_class_df.y.values,'y-coordinates','Probability Density','PDF of Bounding Box Y-coordinates')
plot_cdf(merged_class_df['y'],'Y-coordinates','Y-coordinates','CDF of Bounding Box Y-coordinates')
box_plot(merged_class_df,'y','Boxplot of Bounding Box Y-coordinates')

Observation:

The decline that we can observe at the middle of the curve is due to the centre of the chest as there is no lungs at the centre (and bounding boxes are only on lungs).
The minimum & maximum value that we got is 0 and 800 respectively.
Almost 99% values are between 100 & 700.
The IQR(inerquartile range) we got is from 250 (25th percentile) & 500 (75th percentile).
The median value that we got is 350 (approx).
There are certain outlier values also above 800.

5.3) Bounding Box Width¶

plot_pdf(merged_class_df.width.values,'width','Probability Density','PDF of Bounding Box width')
plot_cdf(merged_class_df['width'],'Width','Width','CDF of Bounding Box width')
box_plot(merged_class_df,'width','Boxplot of Bounding Box width')

Observation:

The PDF of the bounding box width that we got is approx normally distributed.
The minimum & maximum values that we got is 50 & 400 with certain outliers above 400.
Almost 99% values are less than 350.
The IQR(interquartile range) that we got is from 175 to 275 respectively.
The median value that we got is 225 (approx).

5.4) Bounding Box Height¶

plot_pdf(merged_class_df.height.values,'Height','Probability Density','PDF of Bounding Box Height')
plot_cdf(merged_class_df['height'],'Height','Height','CDF of Bounding Box Height')
box_plot(merged_class_df,'height','Boxplot of Bounding Box Height')

Observation:

The PDF of the bounding box height that we got is positively skewed.
The minimum & maximum values that we got is 0 & 800 with certain outliers above 800.
Almost 99% values are less than 700.
The IQR(interquartile range) that we got is from 200 to 450 respectively.
The median value that we got is 300 (approx).

5.5) Bounding Box Area¶

plot_pdf(merged_class_df.width.values * merged_class_df.height.values,'Area','Probability Density','PDF of Bounding Box Area')
plot_cdf(merged_class_df['width'] * merged_class_df['height'],'Area','Area','CDF of Bounding Box Area')

Observation:

The PDF of the bounding box bounding-box area that we got is positively skewed.
The general minimum & maximum values that we got is 0 & 200000px (pixels) respectively.
Almost 99% values are less than 250000.
The IQR(interquartile range) that we got is from 25000 to 100000 respectively.
There are certain outliers above 200000.

5.6) Bounding Box Aspect Ratio¶

plot_pdf(merged_class_df.width.values / merged_class_df.height.values,'Area','Probability Density','PDF of Bounding Box Aspect Ratio')
plot_cdf(merged_class_df['width'] / merged_class_df['height'],'Area','Area','CDF of Bounding Box Aspect Ratio')

Observation:

The PDF of the bounding box bounding-box area that we got is positively skewed.
The general minimum & maximum values that we got is 0 & 2 respectively.
Almost 99% values are less than 2.
There are certain outliers above 1.75.

5.7) Bounding Box centroid¶

centers = (merged_class_df.dropna(subset=['x'])
           .assign(center_x=merged_class_df.x + merged_class_df.width / 2, center_y=merged_class_df.y + merged_class_df.height / 2))

from sklearn.mixture import GaussianMixture
clf = GaussianMixture(n_components=2)
clf.fit(centers[['center_x', 'center_y']])
center_probs = clf.predict_proba(centers[['center_x', 'center_y']])
Z = -clf.score_samples(centers[['center_x', 'center_y']])
outliers = centers.iloc[Z > 17]
fig, ax = plt.subplots(figsize=(15,8))
centers.plot.scatter('center_x', 'center_y', c=Z, alpha=0.5, cmap='viridis', ax=ax)
outliers.plot.scatter('center_x', 'center_y', c='red', marker='x', s=100, ax=ax)
_ = ax.set_title('Detecting Outliers Bounding Boxes', fontsize=18)

Observation:

We can observe that the central area of both the lungs are highly dense, having maximum numbers of bounding box centroids.
As soon as we are moving away from the centre, the bounding box density is gradually decreasing.
The red crosses that we can see on the outskerts of the lungs are outliers.
By removing those outlier values, we can get rid of the effect of outliers.

Data Pre-processing¶

1) Filtering Dataset¶

def filterPatientsByCriteria(criterion):
    x = 1
    for key in criterion.keys():
      if (x==1):
        filt_class_df = det_class_info_df[det_class_info_df['class']==key].sample(int(criterion.get(key)))
      else:
        filt_class_df=filt_class_df.append(det_class_info_df[det_class_info_df['class']==key].sample(int(criterion.get(key))))
      x = x + 1
    return filt_class_df

# Function to generate list of augmentated dataframe
def generateAugmentationList(filt_class_df, augm_count):
  rowlist=[]
  augm_data=[]
  for i in range(augm_count):
    exists=True 
    while exists:
      n=random.randint(0,filt_class_df.shape[0]-1)
      if n in rowlist:
        exists=True
      else:
        rowlist.append(n)
        row = filt_class_df.iloc[n]
        augm_data.append(['xxx ' + row['patientId'], row['class'], row['Target'], 'aug'])
        exists=False
  filt_class_df=filt_class_df.append(pd.DataFrame(augm_data, 
               columns=[ 'patientId', 'class', 'Target', 'row_type']),
               ignore_index = True)
  return filt_class_df

# Function to get filtered and augmented data based on given criterion and augmentation count
def get_filtered_data_criterion(criterion, aug_count):
  filt_class_df = pd.DataFrame()
  if not os.path.exists(data_path + '/' + 'filtered_data_aug.csv'):
    # Filter data based on the criterion
    filt_class_df = filterPatientsByCriteria(criterion)
    # Dropping the checksum from the filtered dataset as it is not relevant
    filt_class_df.drop(['checksum'], axis=1, inplace=True)
    # Add a new column with default value of 'orig'. The augmented row will have row_type = 'aug'
    filt_class_df['row_type'] = 'orig'
    # Append the augmented rows to the list
    filt_class_df=generateAugmentationList(filt_class_df, aug_count) # Augmenting 3000 images from the filtered list
    filt_class_df=shuffle(filt_class_df)
    filt_class_df.reset_index(drop=True)
    # Export the data to CSV
    filt_class_df.to_csv(data_path + '/' + 'filtered_data_aug.csv')
  else:
    filt_class_df = pd.read_csv(data_path + '/' + 'filtered_data_aug.csv')
  return filt_class_df

# Setting up criterion to filter the data
# Set the criterion = {} to load all data
criterion={'Lung Opacity':3000, 
           'No Lung Opacity / Not Normal': 400, 
           'Normal': 400}

# Get Filtered data
filt_class_df = pd.DataFrame()
filt_class_df =  get_filtered_data_criterion(criterion, 3000)

filt_class_df.head()

merged_class_df.head()

# Function to filter the test images 
def filterTestPatientsByCriteria(criterion):
    x = 1
    # Create a list of patients that are not in training dataset
    delta_df = det_class_info_df[~det_class_info_df.patientId.isin(filt_class_df.patientId)]
    for key in criterion.keys():
      if (x==1):
        filt_test_class_df = delta_df[delta_df['class']==key].sample(int(criterion.get(key)))
      else:
        filt_test_class_df=filt_test_class_df.append(delta_df[delta_df['class']==key].sample(int(criterion.get(key))))
      x = x + 1
    return filt_test_class_df

# Criteria for loading test data
criterion={'Lung Opacity':300, 
           'No Lung Opacity / Not Normal': 25, 
           'Normal': 25}

# For the first time, filter the patient data based on given criteria
# Otehrwise Loading the filtered data from CSV if already saved on disk
filt_test_class_df = pd.DataFrame()
if not os.path.exists(data_path + '/' + 'filtered_test_data.csv'):
  filt_test_class_df =  filterTestPatientsByCriteria(criterion)
  filt_test_class_df.to_csv(data_path + '/' + 'filtered_test_data.csv')
else:
  filt_test_class_df = pd.read_csv(data_path + '/' + 'filtered_test_data.csv', index_col=[0])

filt_test_class_df.reset_index(drop=True)

2) Image Augmentation¶

def getBoundingBox(patientId, shape):
  bbs=[]
  for idx_a, row in merged_class_df[(merged_class_df['patientId']==patientId)].iterrows():
    bbs.append(BoundingBox(x1=row['x'], y1=row['y'], x2=row['x']+row['width'], y2=row['y']+row['height']))
  return BoundingBoxesOnImage(bbs, shape=shape)

def getImage(patientId):
  dcm_img = dcm.read_file(data_path +'/' + 'stage_2_train_images' + '/' + patientId + '.dcm').pixel_array
  return dcm_img

def augment(patientId, target):
  dcm_img = getImage(patientId)
  aug_bbs=None
  augmentation = iaa.Sequential([
      iaa.Fliplr(0.5),
      iaa.OneOf([ ## geometric transform
          iaa.Affine(
              scale={"x": (0.98, 1.02), "y": (0.98, 1.04)},
              translate_percent={"x": (-0.02, 0.02), "y": (-0.04, 0.04)},
              rotate=(-2, 2),
              shear=(-1, 1),
          ),
          iaa.PiecewiseAffine(scale=(0.001, 0.025)),
      ]),
      iaa.OneOf([ ## brightness or contrast
          iaa.Multiply((0.9, 1.1)),
          iaa.ContrastNormalization((0.9, 1.1)),
      ]),
      iaa.OneOf([ ## blur or sharpen
          iaa.GaussianBlur(sigma=(0.0, 0.1)),
          iaa.Sharpen(alpha=(0.0, 0.1)),
      ])
  ], random_order=True)

  det = augmentation.to_deterministic()
  aug_image = det.augment_image(dcm_img)
  if target == 1:
    bbs = getBoundingBox(patientId, dcm_img.shape)
    aug_bbs = det.augment_bounding_boxes(bbs)
  return aug_image, aug_bbs

def filter_augment_generator(merged_class_df):
  image_array=[]
  merge_aug_data=[]
  m = 0
  count = 200
  for idx, row in filt_class_df.iterrows():
    m += 1
    if row['row_type']=='orig':
      orig_img=getImage(row['patientId'])
      image_array.append(orig_img)
    else:
      aug_image, aug_bbs = augment(row['patientId'][4: :], row['Target'])
      image_array.append(aug_image)
      if row['Target']==1:
        for i in range(len(aug_bbs.bounding_boxes)):
          aug_bb = aug_bbs.bounding_boxes[i]
          x, y, width, height = aug_bb.x1, aug_bb.y1, aug_bb.x2-aug_bb.x1, aug_bb.y2-aug_bb.y1
          merge_aug_data.append([row['patientId'], x, y, width, height, row['Target'], row['class']])
      else:
        merge_aug_data.append([row['patientId'], np.nan, np.nan, np.nan, np.nan, row['Target'], row['class']])
    if m%count == 0:
      print (str(m))
  merged_class_df=merged_class_df.append(pd.DataFrame(merge_aug_data, 
              columns=[ 'patientId', 'x', 'y', 'width', 'height', 'Target', 'class']),
              ignore_index = True)
  return merged_class_df, np.array(image_array)

Load filtered & augmented Train images

def load_filtered_augmented_images(merged_class_df):
  orig_aug_array=[]
  if not os.path.exists(data_path + '/' + 'filtered_image_array_aug_orig.npz'):
    merged_class_df, orig_aug_array = filter_and_augment(merged_class_df)
    np.savez_compressed(data_path + '/' + 'filtered_image_array_aug_orig.npz', images=orig_aug_array)
    merged_class_df.to_csv(data_path + '/' + 'merged_data_aug.csv')
  else:
    orig_aug_array = np.load(data_path + '/' + 'filtered_image_array_aug_orig.npz', allow_pickle=True)['images']
    if os.path.exists(data_path + '/' + 'merged_data_aug.csv'):
      merged_class_df = pd.read_csv(data_path + '/' + 'merged_data_aug.csv')
  return merged_class_df, orig_aug_array

orig_aug_array=[]
merged_class_df, orig_aug_array = load_filtered_augmented_images(merged_class_df)

orig_aug_array.shape

(6800, 1024, 1024)

merged_class_df.tail()

merged_class_df["patientId"]= merged_class_df["patientId"].str.replace("xxx ", "xxx_", case = True)
filt_class_df["patientId"]= filt_class_df["patientId"].str.replace("xxx ", "xxx_", case = True)
filt_test_class_df["patientId"]= filt_test_class_df["patientId"].str.replace("xxx ", "xxx_", case = True)

Load filtered & augmented Test Images

def loadTestImages():
  test_image_array=[]
  for idx, row in filt_test_class_df.iterrows():
    test_image_array.append(getImage(row['patientId']))
  return np.array(test_image_array)

test_orig_array=[]
if not os.path.exists(data_path + '/' + 'filtered_image_test_orig.npz'):
  test_orig_array=loadTestImages()
  np.savez_compressed(data_path + '/' + 'filtered_image_test_orig.npz', images=test_orig_array)
else:
  test_orig_array = np.load(data_path + '/' + 'filtered_image_test_orig.npz', allow_pickle=True)['images']

test_orig_array.shape

(350, 1024, 1024)

3) Resize Images¶

Resize images to a smaller size to ensure the model training doesn't use up all the RAM

orig_img_size=1024
do_not_resize=True
target_img_size=256
target_mask_size=int(target_img_size/2)

def resizeMask(idx, imgShape):
  res_mask=np.zeros(imgShape)
  patientId = filt_class_df.iloc[idx]['patientId']
  for idx_a, row_a in merged_class_df[(merged_class_df['patientId']==patientId) & (merged_class_df['Target']==1)].iterrows():
    res_mask[int(row_a['y']):int(row_a['y'])+ int(row_a['height']), 
        int(row_a['x']): int(row_a['x']) + int(row_a['width'])] = 1
  res_mask = resize(res_mask, (target_mask_size, target_mask_size), mode='symmetric')
  return res_mask

# Convert each file
def resizeImage(idx):
    try:
        res_array = resize(orig_aug_array[idx], (target_img_size, target_img_size), mode='symmetric')
    except Exception as e:
        print(file)
        print(e)
    return res_array

# Convert all files within a folder
IST=pytz.timezone('Asia/Kolkata')
def resizeImages():
    res_array=[]
    cnt=filt_class_df.shape[0]
    print('Resizing', str(cnt), 'files from size - 1024x1024 to', str(target_img_size) + 'x' + str(target_img_size))
    print ('-----------------------------------------------------')
    current_time = datetime.datetime.now(IST).strftime("%H:%M:%S")
    print("Start Time =", current_time)
    for idx in range(cnt):
      ia = resizeImage(idx)
      res_array.append(ia)
    current_time = datetime.datetime.now(IST).strftime("%H:%M:%S")
    print("End Time =", current_time)
    return np.array(res_array)

#Loading resized image array. 
#If pneumonia_data_orig_aug_res_<>.npz doesn't exist, then resize first and save it 
def load_image():
  resized_array=[]
  if not os.path.exists(data_path + '/' + 'pneumonia_data_orig_aug_res_' + str(target_img_size) + '.npz'):
    resized_array = resizeImages()
    np.savez_compressed(data_path + '/' + 'pneumonia_data_orig_aug_res_' + str(target_img_size) + '.npz', images=resized_array)
  else:
    resized_array = np.load(data_path + '/' + 'pneumonia_data_orig_aug_res_' + str(target_img_size) + '.npz', allow_pickle=True)['images']
  return resized_array

resized_array=[]
if do_not_resize:
  resized_array=orig_aug_array
else:
  resized_array = load_image()
  del orig_aug_array

print(resized_array.shape)

(6800, 1024, 1024)

Resize test images

def load_test_images():
  resized_test_array=[]
  if not os.path.exists(data_path + '/' + 'pneumonia_data_orig_test_' + str(target_img_size) + '.npz'):
    cnt=filt_test_class_df.shape[0]
    for idx in range(cnt):
      ia = resize(test_orig_array[idx], (target_img_size, target_img_size), mode='symmetric')
      resized_test_array.append(ia)
    np.savez_compressed(data_path + '/' + 'pneumonia_data_orig_test_' + str(target_img_size) + '.npz', images=resized_test_array)
  else:
    resized_test_array = np.load(data_path + '/' + 'pneumonia_data_orig_test_' + str(target_img_size) + '.npz', allow_pickle=True)['images']
  return np.array(resized_test_array)

if do_not_resize:
  resized_test_array=test_orig_array
else:
  resized_test_array=load_test_images()
  del test_orig_array

print(resized_test_array.shape)

(350, 1024, 1024)

3) Visualize Images after pre-processing¶

Defining function for plotting images

# This function is to validate the JPG display against the original
def plot_dcm_images(ax, patientId, c, bb_box=False):
  dcm_data = dcm.read_file(data_path +'/' + 'stage_2_train_images/' + patientId + '.dcm')
  ax[c//3, c%3].imshow(dcm_data.pixel_array, cmap=plt.cm.bone)
  ax[c//3, c%3].set_title('ID: {}\n'.format(patientId))
  if bb_box:
    for idx, row in merged_class_df[merged_class_df['patientId']==patientId].iterrows():
      ax[c//3, c%3].add_patch(Rectangle(xy=(row['x'], row['y']),
                                        width=row['width'],height=row['height'],
                                        linewidth=1, edgecolor='r', facecolor='none'))

# Pass bb_box as True for target 1 to draw bounding boxes
def plot_images(ax, patientId, c, bb_box=False, col=3):
  idx=np.where(filt_class_df['patientId']==patientId)
  if do_not_resize:
    target_img_size = orig_img_size
  scale=target_img_size/orig_img_size
  ax[c//col, c%col].imshow(resized_array[idx].reshape(target_img_size, target_img_size), cmap=plt.cm.bone)
  ax[c//col, c%col].set_title('ID: {}\n'.format(patientId))
  if bb_box:
    for idx, row in merged_class_df[merged_class_df['patientId']==patientId].iterrows():
      ax[c//col, c%col].add_patch(Rectangle(xy=(row['x']*scale, row['y']*scale),
                 width=row['width']*scale,height=row['height']*scale,
                 linewidth=2, edgecolor="white",fill=False))

Plot images with bounding boxes of subjects that were detected with Pneumonia¶

c = 0
f, ax = plt.subplots(3,3, figsize=(16,18))
# Array for storing the ids that can be used to display DCM 
image_ids=[]
# Target = 1
#for patientId in filt_class_df[(filt_class_df['Target']==1) & (filt_class_df['patientId'].str.startswith('xxx_')==False)].patientId.sample(9):
for patientId in filt_class_df[(filt_class_df['Target']==1) & (filt_class_df['row_type']=='orig')].patientId.sample(9):
  image_ids.append(patientId)
  plot_images (ax, patientId, c, True)
  c = c + 1

Plot original DCM images¶

c = 0
f, ax = plt.subplots(3,3, figsize=(16,18))
# Target = 1
for patientId in image_ids:
  plot_dcm_images (ax, patientId, c, True)
  c = c + 1

Plot images of subjects with no Pneumonia¶

c = 0
f, ax = plt.subplots(3,3, figsize=(16,18))
# Array for storing the ids that can be used to display DCM 
image_ids1=[]
# Target = 0
for patientId in filt_class_df[(filt_class_df['Target']==0) & (filt_class_df['patientId'].str.startswith('xxx ')==False)].patientId.sample(9):
  image_ids1.append(patientId)
  plot_images (ax, patientId, c, False)
  c = c + 1

Plot augmented images side by side with original¶

c = 0
f, ax = plt.subplots(3,2, figsize=(12,16))
#for patientId in filt_class_df[(filt_class_df['Target']==1) & (filt_class_df['patientId'].str.startswith('xxx ')==True)].patientId.sample(3):
for patientId in filt_class_df[(filt_class_df['Target']==1) & (filt_class_df['row_type']=='aug')].patientId.sample(3):
  plot_images (ax, patientId, c, True, 2)
  c = c + 1
  patientId = patientId[4::]
  plot_images (ax, patientId, c, True, 2)
  c = c + 1

Plot masks for subjects with Pneumonia¶

def getMask(patientId, imgShape):
  image_mask=np.zeros(imgShape)
  if do_not_resize:
    target_mask_size=orig_img_size
  for idx_a, row_a in merged_class_df[(merged_class_df['patientId']==patientId) & (merged_class_df['Target']==1)].iterrows():
    image_mask[int(row_a['y']):int(row_a['y'])+ int(row_a['height']), 
        int(row_a['x']): int(row_a['x']) + int(row_a['width'])] = 1
  image_mask = resize(image_mask, (target_mask_size, target_mask_size), mode='symmetric')
  return image_mask

# Pass bb_box as True for target 1 to draw bounding boxes
def plot_masks(filt_df, ax, patientId, c, bb_box=False):
  idx=np.where(filt_df['patientId']==patientId)
  ax[c//3, c%3].imshow(getMask(patientId, (1024, 1024)))
  ax[c//3, c%3].set_title('ID: {}\n'.format(patientId))

c = 0
f, ax = plt.subplots(3,3, figsize=(16,18))
# Target = 1
for patientId in image_ids:
  plot_masks (filt_class_df, ax, patientId, c, True)
  c = c + 1

Model Training & Evaluation¶

1) Base classes¶

ROOT_DIR=data_path
import sys, math
sys.path.append(os.path.join(lib_path, 'Mask_RCNN'))

class MaskRCnnConfig(config.Config):
    NAME = 'MaskRCNN'
    TRAINING_WEIGHTS=lib_path + '/' + 'Mask_RCNN' + '/' + 'mask_rcnn_coco.h5'
    TESTING_WEIGHTS=''
    BACKBONE = 'resnet50'
    NUM_CLASSES = 2
    IMAGES_PER_GPU=2
    ORIG_IMAGE_SHAPE=(1024, 1024)
    OPTIMIZER='SGD'
#    MASK_SHAPE=(28, 28)
    IMAGE_MIN_DIM = 256
    IMAGE_MAX_DIM = 256
    MEAN_PIXEL=[126.7,71.5,130.5]
    RPN_ANCHOR_SCALES = (16, 32, 64, 128)
    TRAIN_ROIS_PER_IMAGE = 32
    MAX_GT_INSTANCES = 4
    DETECTION_MAX_INSTANCES = 4
    DETECTION_MIN_CONFIDENCE = 0.78
    DETECTION_NMS_THRESHOLD = 0.01
    STEPS_PER_EPOCH = 100
    BACKBONE_STRIDES = [4, 8, 16, 32, 64]
    VALIDATION_STEPS=50
    USE_MINI_MASK=True

class InferenceConfig(MaskRCnnConfig):
    GPU_COUNT = 1
    USE_MINI_MASK=False
    IMAGES_PER_GPU = 1

class MaskDataset(utils.Dataset):
    def __init__(self, image_fps, image_annotations, config, image_array):
        super().__init__(self)
        # Add classes
        self.config=config
        self.image_array=image_array
        self.image_annotations = image_annotations
        self.add_class('pneumonia', 1, 'Lung Opacity')
        
        # add images 
        for i, patiendId in enumerate(image_fps):
            annotations = self.image_annotations[patiendId]
            self.add_image('pneumonia', image_id=i, annotations=annotations, image_height=self.config.ORIG_IMAGE_SHAPE[0], image_width=self.config.ORIG_IMAGE_SHAPE[1])

    def add_image(self, source, image_id, **kwargs):
        image_info = {
            "id": image_id,
            "source": source,
        }
        image_info.update(kwargs)
        self.image_info.append(image_info)

    def load_image(self, image_id):
        image = self.image_array[image_id]
#        image = apply_color_lut(image, palette='PET')
        if len(image.shape) != 3 or image.shape[2] != 3:
            image = np.stack((image,) * 3, -1)
        return image

    def load_mask(self, image_id):
        info = self.image_info[image_id]
        annotations = info['annotations']
        count = len(annotations)
        if count == 0:
            mask = np.zeros((info['image_height'], info['image_width'], 1), dtype=np.uint8)
            class_ids = np.zeros((1,), dtype=np.int32)
        else:
            mask = np.zeros((info['image_height'], info['image_width'], count), dtype=np.uint8)
            class_ids = np.zeros((count,), dtype=np.int32)
            for i, a in enumerate(annotations):
                if a['Target'] == 1:
                    x = int(a['x'])
                    y = int(a['y'])
                    w = int(a['width'])
                    h = int(a['height'])
                    mask_instance = mask[:, :, i].copy()
                    cv2.rectangle(mask_instance, (x, y), (x+w, y+h), 255, -1)
                    mask[:, :, i] = mask_instance
                    class_ids[i] = 1
        return mask.astype(np.bool), class_ids.astype(np.int32)

2) Define Mask RCNN Model & Functions¶

# Dictionary of results keyed by id.
model_results={}
# Training models keyed by id.
model_list={}
# Track epoch
epoch_list={}
# Iteration Id - Maps the static to dynamic id used for training and tracking
iteration_list={}
#history
history_list={}

def get_log_dir(static_id):
  return data_path + '/' + 'logs/maskrcnn' + '/' + iteration_list[static_id] + '/'

def get_checkpoint_weight_dir(static_id):
  return data_path + '/' + 'checkpoint/maskrcnn' + '/' + iteration_list[static_id] + '/'

def get_checkpoint_weight_path(static_id, epochid):
  return get_checkpoint_weight_dir(static_id) + 'maskrcnn_' + str(epochid).zfill(4) + '.h5'

def clear_runs(static_id, dir=False):
  if static_id in iteration_list:
    id = iteration_list[static_id]
    if id in model_list:
      model_list.pop(iteration_list[static_id])
    if id in epoch_list:
      epoch_list.pop(iteration_list[static_id])
    if id in model_results:
      model_results.pop(iteration_list[static_id])
    if dir:
      shutil.rmtree(get_log_dir(static_id))
      shutil.rmtree(get_checkpoint_weight_dir(static_id))
    if static_id in iteration_list:
      iteration_list.pop(static_id)
  tf.keras.backend.clear_session()

def get_patient_list(data_df):
  return data_df['patientId'].tolist()

def parse_dataset(labels, data_df): 
  image_annotations={}
  image_list = get_patient_list(data_df)
  image_annotations = {p: [] for p in image_list}
  for patientId in image_list:
    for index, row in merged_class_df[merged_class_df['patientId']==patientId][['x', 'y', 'width', 'height', 'Target']].iterrows(): 
      image_annotations[patientId].append(row)
  return image_list, image_annotations

def reset_weights(id, model, weights_path):
  model.load_weights(id, weights_path, by_name=True, exclude=["mrcnn_class_logits", 
                                                          "mrcnn_bbox_fc", 
                                                          "mrcnn_bbox", 
                                                          "mrcnn_mask"])
  return model

def create_maskmodel(id, config):
  model = modellib.MaskRCNN(id=id, mode='training', config=config, model_dir=ROOT_DIR)
  weights_path = mask_config.TRAINING_WEIGHTS
  # Exclude the last layers because they require a matching
  # number of classes
  model.load_weights(id, weights_path, by_name=True, exclude=["mrcnn_class_logits", 
                                                              "mrcnn_bbox_fc",
                                                              "mrcnn_bbox",
                                                              "mrcnn_mask"])
  return model

def create_inference_maskmodel(static_id, config):
  if static_id in iteration_list:
    id = iteration_list[static_id]
    model_path = config.TESTING_WEIGHTS
    model = modellib.MaskRCNN(id=id,
                              mode='inference', 
                              config=config,
                              model_dir=ROOT_DIR)
    model.load_weights(id, model_path, by_name=True)
    return model

def get_best_epoch(static_id, metric, type="min"):
  id = iteration_list[static_id]
  if type=="min":
    return model_results[id][model_results[id][metric]==model_results[id][metric].min()]
  else:
    return model_results[id][model_results[id][metric]==model_results[id][metric].max()]

def plot_graph(data, ):
  plt.figure(figsize=(22,5))
  plt.subplot(131)
  plt.plot(data.index, data["loss"], label="Train loss")
  plt.plot(data.index, data["val_loss"], label="Valid loss")
  plt.legend()
  plt.subplot(132)
  plt.plot(data.index, data["mrcnn_class_loss"], label="Train class loss")
  plt.plot(data.index, data["val_mrcnn_class_loss"], label="Valid class loss")
  plt.legend()
  plt.subplot(133)
  plt.plot(data.index, data["mrcnn_bbox_loss"], label="Train box loss")
  plt.plot(data.index, data["val_mrcnn_bbox_loss"], label="Valid box loss")
  plt.legend()
  plt.show()

def plot_loss_graph(static_id):
  plt.figure(figsize=(22,5))
  id = iteration_list[static_id]
  plot_graph(model_results[id])

def initialize_wandb(name, config):
  config_dict = mask_config.to_dict()
  coi = ['BACKBONE', 'LEARNING_MOMENTUM', 'LEARNING_RATE',
          'WEIGHT_DECAY', 'STEPS_PER_EPOCH', 'OPTIMIZER', 'LAYERS']
  run = wandb.init(name=name, 
                   project="Capstone AIML",
                   config=({k: config_dict[k] for k in coi})
                   )
  return run

def add_iteration(static_id, iteration_id):
  iteration_list[static_id]=iteration_id

def get_iteration_id(static_id):
  def generate_id(static_id):
    now = datetime.datetime.now()
    return str(static_id) +"_{:%Y%m%dT%H%M}".format(now)
  if static_id in iteration_list:
    iteration_id=iteration_list[static_id]
  else:
    iteration_id = generate_id(static_id)
    iteration_list[static_id] = iteration_id
  print('Iteration Id:',iteration_id)
  return iteration_id

def get_nn_model(static_id, config, optimizer):
  global model_list, data_path
  id = iteration_list[static_id]
  if id in model_list:
    model=model_list[id]
    weights_path = config.TRAINING_WEIGHTS
    model = reset_weights(id, model, weights_path)  
    return model
  else:
    model=create_maskmodel(id, config)
    model.construct(learning_rate=config.LEARNING_RATE, layers=config.LAYERS, optim=optimizer)
    model_list[id]=model
    model=model_list[id]
    return model

def add_model_results(static_id):
  id = iteration_list[static_id]
  if id in history_list:
    hist_list = history_list[id]
    i = 0
    model_result = pd.DataFrame()
    for hist in hist_list:
      result = pd.DataFrame(hist)
      model_result = model_result.append(result, ignore_index=True)
    model_results[id]=model_result

def add_best_epoch_to_list(static_id, metric, type):
  id = iteration_list[static_id]
  best_epoch = get_best_epoch(static_id, metric, type)
  epoch_list[id]=best_epoch.index[0]

def add_history_to_list(static_id, history):
  id = iteration_list[static_id]
  if id in history_list:
    hist_obj = history_list[id]
    hist_obj.append(history)
    history_list[id] = hist_obj
  else:
    history_list[id] = [history]

def save_run_state(run_id, static_id):
  run_path = data_path + '/' + 'runs/maskrcnn/' + str(run_id)
  with open(os.path.join(run_path, static_id + '_iteration_list.pkl'), 'wb') as iter_list_file:
    pickle.dump(iteration_list, iter_list_file)
  with open(os.path.join(run_path, static_id + '_history_list.pkl'), 'wb') as history_list_file:
    pickle.dump(history_list, history_list_file)
  with open(os.path.join(run_path, static_id + '_model_results.pkl'), 'wb') as model_results_file:
    pickle.dump(model_results, model_results_file)
  with open(os.path.join(run_path, static_id + '_epoch_list.pkl'), 'wb') as epoch_list_file:
    pickle.dump(epoch_list, epoch_list_file)

def load_state(run_id, static_id):
  global iteration_list, model_list, history_list, model_results, epoch_list, ITERATION_MASTER_LIST
  run_path = data_path + '/' + 'runs/maskrcnn/' + str(run_id)
  if os.path.exists(os.path.join(run_path, static_id + '_iteration_list.pkl')):
    with open(os.path.join(run_path, static_id + '_iteration_list.pkl'), 'rb') as iter_list_file:
      iteration_list=pickle.load(iter_list_file)
    with open(os.path.join(run_path, static_id + '_history_list.pkl'), 'rb') as history_list_file:
      history_list=pickle.load(history_list_file)
    with open(os.path.join(run_path, static_id + '_model_results.pkl'), 'rb') as model_results_file:
      model_results=pickle.load(model_results_file)
    with open(os.path.join(run_path, static_id + '_epoch_list.pkl'), 'rb') as epoch_list_file:
      epoch_list=pickle.load(epoch_list_file)

def load_prior_run_state(run_id, static_id):
# Check the index of the current static id to be run.
# Load previous static id state
  if ITERATION_MASTER_LIST.index(static_id) > 0:
    static_id = ITERATION_MASTER_LIST[ITERATION_MASTER_LIST.index(static_id)-1]
    load_state(run_id, static_id)

3) Split data into training and validation¶

image_list, image_annotations = parse_dataset(labels_df, filt_class_df)

test_image_list, test_image_annotations=parse_dataset(labels_df, filt_test_class_df)

X_train, X_val = train_test_split(image_list, test_size=0.15, random_state=0)

Prepare training dataset

mask_config = MaskRCnnConfig()
dataset_train = MaskDataset(X_train, image_annotations, mask_config, resized_array)
dataset_train.prepare()

Prepare validation dataset

mask_config = MaskRCnnConfig()
dataset_val = MaskDataset(X_val, image_annotations, mask_config, resized_array)
dataset_val.prepare()

Prepare test dataset

maskrcnn_infer_config = InferenceConfig()
dataset_test = MaskDataset(test_image_list, test_image_annotations, maskrcnn_infer_config, resized_test_array)
dataset_test.prepare()

4) Training Iterations¶

%load_ext tensorboard

Training Configuration¶

# It is mandatory to add all iteration static ids in this list
ITERATION_MASTER_LIST=['MR_1', 'MR_2', 'MR_3', 'MR_4', 'MR_5', 'MR_5A','MR_6', 'MR_6A', 'MR_6B','MR_7', 'MR_7A', 'MR_7B','MR_8', 'MR_8A', 'MR_8B']

# Set the run id to the specific id if you want to re-run from a specific one.
# Else leave it None
RUN_ID='run_20210605T1740'

if RUN_ID is None:
  now = datetime.datetime.now()
  os.makedirs(os.path.join(data_path, 'runs', 'maskrcnn', "run_{:%Y%m%dT%H%M}".format(now)))
  RUN_ID = "run_{:%Y%m%dT%H%M}".format(now)

print ("Run ID:", RUN_ID)
print ("State will be saved in:", data_path +  '/runs/maskrcnn/' + RUN_ID)

Run ID: run_20210605T1740
State will be saved in: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/runs/maskrcnn/run_20210605T1740

Determine the LR and Image type¶

Iteration 1) Determine the best learning rate using LRFinder¶

STATIC_ID = 'MR_1'

load_prior_run_state(RUN_ID, STATIC_ID)

# Iteration specific configuration
class MaskConfig_1(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  STEPS_PER_EPOCH=723
  VALIDATION_STEPS=128
  LEARNING_RATE=1e-3
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_1()
ID = get_iteration_id(STATIC_ID)
optimizer = SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

mask_config.display()

Configurations:
BACKBONE                       resnet50
BACKBONE_STRIDES               [4, 8, 16, 32, 64]
BATCH_SIZE                     8
BBOX_STD_DEV                   [0.1 0.1 0.2 0.2]
COMPUTE_BACKBONE_SHAPE         None
DETECTION_MAX_INSTANCES        4
DETECTION_MIN_CONFIDENCE       0.78
DETECTION_NMS_THRESHOLD        0.01
FPN_CLASSIF_FC_LAYERS_SIZE     1024
GPU_COUNT                      1
GRADIENT_CLIP_NORM             5.0
IMAGES_PER_GPU                 8
IMAGE_CHANNEL_COUNT            3
IMAGE_MAX_DIM                  256
IMAGE_META_SIZE                14
IMAGE_MIN_DIM                  256
IMAGE_MIN_SCALE                0
IMAGE_RESIZE_MODE              square
IMAGE_SHAPE                    [256 256   3]
LAYERS                         heads
LEARNING_MOMENTUM              0.9
LEARNING_RATE                  0.001
LOSS_WEIGHTS                   {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
MASK_POOL_SIZE                 14
MASK_SHAPE                     [28, 28]
MAX_GT_INSTANCES               4
MEAN_PIXEL                     [122.9, 122.9, 122.9]
MINI_MASK_SHAPE                (56, 56)
NAME                           MaskRCNN
NUM_CLASSES                    2
OPTIMIZER                      SGD
ORIG_IMAGE_SHAPE               (1024, 1024)
POOL_SIZE                      7
POST_NMS_ROIS_INFERENCE        1000
POST_NMS_ROIS_TRAINING         2000
PRE_NMS_LIMIT                  6000
ROI_POSITIVE_RATIO             0.33
RPN_ANCHOR_RATIOS              [0.5, 1, 2]
RPN_ANCHOR_SCALES              (16, 32, 64, 128)
RPN_ANCHOR_STRIDE              1
RPN_BBOX_STD_DEV               [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD              0.7
RPN_TRAIN_ANCHORS_PER_IMAGE    256
STEPS_PER_EPOCH                723
TESTING_WEIGHTS                
TOP_DOWN_PYRAMID_SIZE          256
TRAINING_WEIGHTS               /content/drive/MyDrive/Colab/CapstoneProject/dataset/Mask_RCNN/mask_rcnn_coco.h5
TRAIN_BN                       False
TRAIN_ROIS_PER_IMAGE           32
USE_MINI_MASK                  True
USE_RPN_ROIS                   True
VALIDATION_STEPS               128
WEIGHT_DECAY                   0.0001

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_1_20210605T1740
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_1_20210605T1740

Initilize LRFinder parameters

lrfinder = lr_finder.LR_Finder(model)
lrfinder.find(dataset_train, dataset_val, data_path, 1e-6, 1, epochs=1)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_1_20210605T1740/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
723/723 [==============================] - 239s 307ms/step - batch: 277.5000 - size: 8.0000 - loss: 2.2812 - rpn_class_loss: 0.0202 - rpn_bbox_loss: 0.4670 - mrcnn_class_loss: 0.2255 - mrcnn_bbox_loss: 0.3493 - mrcnn_mask_loss: 0.5174 - dice_coeff: 0.7018

plt.figure(figsize=(20, 5))
lrfinder.plot_loss(n_skip_beginning=30, n_skip_end=9)

Learning rate of 1e-3 to 1e-2 seems to be the best learning rate

save_run_state(RUN_ID, STATIC_ID)

Iteration 2) Stack single channel to 3 channels¶

clear_runs('MR_2', True)

STATIC_ID='MR_2'

load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_2(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=80
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_2()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_2_20210606T0519

mask_config.display()

Configurations:
BACKBONE                       resnet50
BACKBONE_STRIDES               [4, 8, 16, 32, 64]
BATCH_SIZE                     8
BBOX_STD_DEV                   [0.1 0.1 0.2 0.2]
COMPUTE_BACKBONE_SHAPE         None
DETECTION_MAX_INSTANCES        4
DETECTION_MIN_CONFIDENCE       0.78
DETECTION_NMS_THRESHOLD        0.01
FPN_CLASSIF_FC_LAYERS_SIZE     1024
GPU_COUNT                      1
GRADIENT_CLIP_NORM             5.0
IMAGES_PER_GPU                 8
IMAGE_CHANNEL_COUNT            3
IMAGE_MAX_DIM                  256
IMAGE_META_SIZE                14
IMAGE_MIN_DIM                  256
IMAGE_MIN_SCALE                0
IMAGE_RESIZE_MODE              square
IMAGE_SHAPE                    [256 256   3]
LAYERS                         heads
LEARNING_MOMENTUM              0.9
LEARNING_RATE                  0.001
LOSS_WEIGHTS                   {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
MASK_POOL_SIZE                 14
MASK_SHAPE                     [28, 28]
MAX_GT_INSTANCES               4
MEAN_PIXEL                     [126.7, 71.5, 130.5]
MINI_MASK_SHAPE                (56, 56)
NAME                           MaskRCNN
NUM_CLASSES                    2
OPTIMIZER                      SGD
ORIG_IMAGE_SHAPE               (1024, 1024)
POOL_SIZE                      7
POST_NMS_ROIS_INFERENCE        1000
POST_NMS_ROIS_TRAINING         2000
PRE_NMS_LIMIT                  6000
ROI_POSITIVE_RATIO             0.33
RPN_ANCHOR_RATIOS              [0.5, 1, 2]
RPN_ANCHOR_SCALES              (16, 32, 64, 128)
RPN_ANCHOR_STRIDE              1
RPN_BBOX_STD_DEV               [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD              0.7
RPN_TRAIN_ANCHORS_PER_IMAGE    256
STEPS_PER_EPOCH                200
TESTING_WEIGHTS                
TOP_DOWN_PYRAMID_SIZE          256
TRAINING_WEIGHTS               /content/drive/MyDrive/Colab/CapstoneProject/dataset/Mask_RCNN/mask_rcnn_coco.h5
TRAIN_BN                       False
TRAIN_ROIS_PER_IMAGE           32
USE_MINI_MASK                  True
USE_RPN_ROIS                   True
VALIDATION_STEPS               80
WEIGHT_DECAY                   0.0001

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_2_20210606T0519
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_2_20210606T0519

#run=initialize_wandb(ID, mask_config)

EPOCHS=5
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_2_20210606T0519/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
Epoch 1/5
200/200 [==============================] - 177s 785ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.7080 - rpn_class_loss: 0.0262 - rpn_bbox_loss: 0.4580 - mrcnn_class_loss: 0.3401 - mrcnn_bbox_loss: 0.6486 - mrcnn_mask_loss: 0.4819 - dice_coeff: 0.7531 - val_loss: 2.7270 - val_rpn_class_loss: 0.0258 - val_rpn_bbox_loss: 0.4872 - val_mrcnn_class_loss: 0.3738 - val_mrcnn_bbox_loss: 0.6100 - val_mrcnn_mask_loss: 0.4787 - val_dice_coeff: 0.7515
Epoch 2/5
200/200 [==============================] - 113s 567ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6180 - rpn_class_loss: 0.0236 - rpn_bbox_loss: 0.4350 - mrcnn_class_loss: 0.3258 - mrcnn_bbox_loss: 0.6038 - mrcnn_mask_loss: 0.4710 - dice_coeff: 0.7588 - val_loss: 2.6589 - val_rpn_class_loss: 0.0229 - val_rpn_bbox_loss: 0.4694 - val_mrcnn_class_loss: 0.3293 - val_mrcnn_bbox_loss: 0.6050 - val_mrcnn_mask_loss: 0.4974 - val_dice_coeff: 0.7350
Epoch 3/5
200/200 [==============================] - 114s 573ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6446 - rpn_class_loss: 0.0231 - rpn_bbox_loss: 0.4807 - mrcnn_class_loss: 0.3164 - mrcnn_bbox_loss: 0.5941 - mrcnn_mask_loss: 0.4767 - dice_coeff: 0.7536 - val_loss: 2.6124 - val_rpn_class_loss: 0.0223 - val_rpn_bbox_loss: 0.4566 - val_mrcnn_class_loss: 0.3110 - val_mrcnn_bbox_loss: 0.5934 - val_mrcnn_mask_loss: 0.4542 - val_dice_coeff: 0.7751
Epoch 4/5
200/200 [==============================] - 123s 619ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6414 - rpn_class_loss: 0.0241 - rpn_bbox_loss: 0.4879 - mrcnn_class_loss: 0.3236 - mrcnn_bbox_loss: 0.5780 - mrcnn_mask_loss: 0.4693 - dice_coeff: 0.7584 - val_loss: 2.6057 - val_rpn_class_loss: 0.0231 - val_rpn_bbox_loss: 0.4630 - val_mrcnn_class_loss: 0.3121 - val_mrcnn_bbox_loss: 0.5799 - val_mrcnn_mask_loss: 0.4627 - val_dice_coeff: 0.7650
Epoch 5/5
200/200 [==============================] - 118s 591ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5853 - rpn_class_loss: 0.0221 - rpn_bbox_loss: 0.4391 - mrcnn_class_loss: 0.3250 - mrcnn_bbox_loss: 0.5725 - mrcnn_mask_loss: 0.4693 - dice_coeff: 0.7573 - val_loss: 2.6456 - val_rpn_class_loss: 0.0231 - val_rpn_bbox_loss: 0.4653 - val_mrcnn_class_loss: 0.3525 - val_mrcnn_bbox_loss: 0.5794 - val_mrcnn_mask_loss: 0.4670 - val_dice_coeff: 0.7584

add_model_results(STATIC_ID)
save_run_state(RUN_ID, STATIC_ID)

Iteration 3) Convert the image to RGB¶

del model
clear_runs('MR_3', True)

STATIC_ID='MR_3'

load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_3(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=80
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_3()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_3_20210606T0555

mask_config.display()

Configurations:
BACKBONE                       resnet50
BACKBONE_STRIDES               [4, 8, 16, 32, 64]
BATCH_SIZE                     8
BBOX_STD_DEV                   [0.1 0.1 0.2 0.2]
COMPUTE_BACKBONE_SHAPE         None
DETECTION_MAX_INSTANCES        4
DETECTION_MIN_CONFIDENCE       0.78
DETECTION_NMS_THRESHOLD        0.01
FPN_CLASSIF_FC_LAYERS_SIZE     1024
GPU_COUNT                      1
GRADIENT_CLIP_NORM             5.0
IMAGES_PER_GPU                 8
IMAGE_CHANNEL_COUNT            3
IMAGE_MAX_DIM                  256
IMAGE_META_SIZE                14
IMAGE_MIN_DIM                  256
IMAGE_MIN_SCALE                0
IMAGE_RESIZE_MODE              square
IMAGE_SHAPE                    [256 256   3]
LAYERS                         heads
LEARNING_MOMENTUM              0.9
LEARNING_RATE                  0.001
LOSS_WEIGHTS                   {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
MASK_POOL_SIZE                 14
MASK_SHAPE                     [28, 28]
MAX_GT_INSTANCES               4
MEAN_PIXEL                     [126.7, 71.5, 130.5]
MINI_MASK_SHAPE                (56, 56)
NAME                           MaskRCNN
NUM_CLASSES                    2
OPTIMIZER                      SGD
ORIG_IMAGE_SHAPE               (1024, 1024)
POOL_SIZE                      7
POST_NMS_ROIS_INFERENCE        1000
POST_NMS_ROIS_TRAINING         2000
PRE_NMS_LIMIT                  6000
ROI_POSITIVE_RATIO             0.33
RPN_ANCHOR_RATIOS              [0.5, 1, 2]
RPN_ANCHOR_SCALES              (16, 32, 64, 128)
RPN_ANCHOR_STRIDE              1
RPN_BBOX_STD_DEV               [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD              0.7
RPN_TRAIN_ANCHORS_PER_IMAGE    256
STEPS_PER_EPOCH                200
TESTING_WEIGHTS                
TOP_DOWN_PYRAMID_SIZE          256
TRAINING_WEIGHTS               /content/drive/MyDrive/Colab/CapstoneProject/dataset/Mask_RCNN/mask_rcnn_coco.h5
TRAIN_BN                       False
TRAIN_ROIS_PER_IMAGE           32
USE_MINI_MASK                  True
USE_RPN_ROIS                   True
VALIDATION_STEPS               80
WEIGHT_DECAY                   0.0001

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_3_20210606T0555
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_3_20210606T0555

#run=initialize_wandb(ID, mask_config)

EPOCHS=5
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_3_20210606T0555/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
Epoch 1/5
200/200 [==============================] - 234s 1s/step - batch: 99.5000 - size: 8.0000 - loss: 2.7608 - rpn_class_loss: 0.0274 - rpn_bbox_loss: 0.4784 - mrcnn_class_loss: 0.3616 - mrcnn_bbox_loss: 0.6535 - mrcnn_mask_loss: 0.4949 - dice_coeff: 0.7450 - val_loss: 2.6415 - val_rpn_class_loss: 0.0239 - val_rpn_bbox_loss: 0.4579 - val_mrcnn_class_loss: 0.3231 - val_mrcnn_bbox_loss: 0.6042 - val_mrcnn_mask_loss: 0.4958 - val_dice_coeff: 0.7367
Epoch 2/5
200/200 [==============================] - 151s 756ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5969 - rpn_class_loss: 0.0245 - rpn_bbox_loss: 0.4287 - mrcnn_class_loss: 0.3165 - mrcnn_bbox_loss: 0.5958 - mrcnn_mask_loss: 0.4743 - dice_coeff: 0.7569 - val_loss: 2.6578 - val_rpn_class_loss: 0.0249 - val_rpn_bbox_loss: 0.4327 - val_mrcnn_class_loss: 0.3534 - val_mrcnn_bbox_loss: 0.6153 - val_mrcnn_mask_loss: 0.4674 - val_dice_coeff: 0.7641
Epoch 3/5
200/200 [==============================] - 161s 809ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5777 - rpn_class_loss: 0.0237 - rpn_bbox_loss: 0.4193 - mrcnn_class_loss: 0.3152 - mrcnn_bbox_loss: 0.5894 - mrcnn_mask_loss: 0.4673 - dice_coeff: 0.7629 - val_loss: 2.6210 - val_rpn_class_loss: 0.0237 - val_rpn_bbox_loss: 0.4313 - val_mrcnn_class_loss: 0.3272 - val_mrcnn_bbox_loss: 0.6105 - val_mrcnn_mask_loss: 0.4701 - val_dice_coeff: 0.7582
Epoch 4/5
200/200 [==============================] - 161s 806ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5798 - rpn_class_loss: 0.0232 - rpn_bbox_loss: 0.4258 - mrcnn_class_loss: 0.3162 - mrcnn_bbox_loss: 0.5849 - mrcnn_mask_loss: 0.4693 - dice_coeff: 0.7604 - val_loss: 2.6133 - val_rpn_class_loss: 0.0221 - val_rpn_bbox_loss: 0.4515 - val_mrcnn_class_loss: 0.3121 - val_mrcnn_bbox_loss: 0.5985 - val_mrcnn_mask_loss: 0.4564 - val_dice_coeff: 0.7727
Epoch 5/5
200/200 [==============================] - 155s 779ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5943 - rpn_class_loss: 0.0234 - rpn_bbox_loss: 0.4636 - mrcnn_class_loss: 0.3037 - mrcnn_bbox_loss: 0.5752 - mrcnn_mask_loss: 0.4682 - dice_coeff: 0.7603 - val_loss: 2.6183 - val_rpn_class_loss: 0.0228 - val_rpn_bbox_loss: 0.4822 - val_mrcnn_class_loss: 0.3040 - val_mrcnn_bbox_loss: 0.5803 - val_mrcnn_mask_loss: 0.4550 - val_dice_coeff: 0.7741

add_model_results(STATIC_ID)
save_run_state(RUN_ID, STATIC_ID)

Iteration 4) Use Original Image¶

STATIC_ID='MR_4'

load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_4(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.001
  IMAGE_SHAPE=[1024,1024,3]
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=80
  IMAGE_MAX_DIM=1024
  IMAGE_MIN_DIM=1024
  RPN_TRAIN_ANCHORS_PER_IMAGE=512
  MINI_MASK_SHAPE=(256, 256)
  MEAN_PIXEL=np.array([510, 286, 522])
  RPN_ANCHOR_SCALES=(32, 64, 128, 256)
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_4()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_4_20210606T0648

 mask_config.display()

Configurations:
BACKBONE                       resnet50
BACKBONE_STRIDES               [4, 8, 16, 32, 64]
BATCH_SIZE                     8
BBOX_STD_DEV                   [0.1 0.1 0.2 0.2]
COMPUTE_BACKBONE_SHAPE         None
DETECTION_MAX_INSTANCES        4
DETECTION_MIN_CONFIDENCE       0.78
DETECTION_NMS_THRESHOLD        0.01
FPN_CLASSIF_FC_LAYERS_SIZE     1024
GPU_COUNT                      1
GRADIENT_CLIP_NORM             5.0
IMAGES_PER_GPU                 8
IMAGE_CHANNEL_COUNT            3
IMAGE_MAX_DIM                  1024
IMAGE_META_SIZE                14
IMAGE_MIN_DIM                  1024
IMAGE_MIN_SCALE                0
IMAGE_RESIZE_MODE              square
IMAGE_SHAPE                    [1024 1024    3]
LAYERS                         heads
LEARNING_MOMENTUM              0.9
LEARNING_RATE                  0.001
LOSS_WEIGHTS                   {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
MASK_POOL_SIZE                 14
MASK_SHAPE                     [28, 28]
MAX_GT_INSTANCES               4
MEAN_PIXEL                     [510 286 522]
MINI_MASK_SHAPE                (256, 256)
NAME                           MaskRCNN
NUM_CLASSES                    2
OPTIMIZER                      SGD
ORIG_IMAGE_SHAPE               (1024, 1024)
POOL_SIZE                      7
POST_NMS_ROIS_INFERENCE        1000
POST_NMS_ROIS_TRAINING         2000
PRE_NMS_LIMIT                  6000
ROI_POSITIVE_RATIO             0.33
RPN_ANCHOR_RATIOS              [0.5, 1, 2]
RPN_ANCHOR_SCALES              (32, 64, 128, 256)
RPN_ANCHOR_STRIDE              1
RPN_BBOX_STD_DEV               [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD              0.7
RPN_TRAIN_ANCHORS_PER_IMAGE    512
STEPS_PER_EPOCH                200
TESTING_WEIGHTS                
TOP_DOWN_PYRAMID_SIZE          256
TRAINING_WEIGHTS               /content/drive/MyDrive/Colab/CapstoneProject/dataset/Mask_RCNN/mask_rcnn_coco.h5
TRAIN_BN                       False
TRAIN_ROIS_PER_IMAGE           32
USE_MINI_MASK                  True
USE_RPN_ROIS                   True
VALIDATION_STEPS               80
WEIGHT_DECAY                   0.0001

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_4_20210606T0648
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_4_20210606T0648

#run=initialize_wandb(ID, mask_config)

EPOCHS=5
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_4_20210606T0648/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 261120, 4)       8355840     input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 256, 256, No 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 53,018,782
Trainable params: 21,069,086
Non-trainable params: 31,949,696
__________________________________________________________________________________________________
Epoch 1/5
200/200 [==============================] - 459s 2s/step - batch: 99.5000 - size: 8.0000 - loss: 3.1234 - rpn_class_loss: 0.0252 - rpn_bbox_loss: 0.8111 - mrcnn_class_loss: 0.3835 - mrcnn_bbox_loss: 0.6624 - mrcnn_mask_loss: 0.5285 - dice_coeff: 0.7126 - val_loss: 3.0697 - val_rpn_class_loss: 0.0231 - val_rpn_bbox_loss: 0.8051 - val_mrcnn_class_loss: 0.3856 - val_mrcnn_bbox_loss: 0.6211 - val_mrcnn_mask_loss: 0.5134 - val_dice_coeff: 0.7214
Epoch 2/5
200/200 [==============================] - 344s 2s/step - batch: 99.5000 - size: 8.0000 - loss: 3.0673 - rpn_class_loss: 0.0224 - rpn_bbox_loss: 0.7964 - mrcnn_class_loss: 0.4007 - mrcnn_bbox_loss: 0.6125 - mrcnn_mask_loss: 0.5057 - dice_coeff: 0.7296 - val_loss: 3.3486 - val_rpn_class_loss: 0.0254 - val_rpn_bbox_loss: 1.2135 - val_mrcnn_class_loss: 0.2532 - val_mrcnn_bbox_loss: 0.6153 - val_mrcnn_mask_loss: 0.4991 - val_dice_coeff: 0.7422
Epoch 3/5
200/200 [==============================] - 342s 2s/step - batch: 99.5000 - size: 8.0000 - loss: 3.0478 - rpn_class_loss: 0.0231 - rpn_bbox_loss: 0.7870 - mrcnn_class_loss: 0.3992 - mrcnn_bbox_loss: 0.6044 - mrcnn_mask_loss: 0.5010 - dice_coeff: 0.7331 - val_loss: 3.0653 - val_rpn_class_loss: 0.0210 - val_rpn_bbox_loss: 0.7951 - val_mrcnn_class_loss: 0.4230 - val_mrcnn_bbox_loss: 0.5939 - val_mrcnn_mask_loss: 0.4884 - val_dice_coeff: 0.7440
Epoch 4/5
200/200 [==============================] - 383s 2s/step - batch: 99.5000 - size: 8.0000 - loss: 3.0182 - rpn_class_loss: 0.0216 - rpn_bbox_loss: 0.7443 - mrcnn_class_loss: 0.4153 - mrcnn_bbox_loss: 0.6034 - mrcnn_mask_loss: 0.4983 - dice_coeff: 0.7353 - val_loss: 2.9843 - val_rpn_class_loss: 0.0205 - val_rpn_bbox_loss: 0.7421 - val_mrcnn_class_loss: 0.3967 - val_mrcnn_bbox_loss: 0.5925 - val_mrcnn_mask_loss: 0.5154 - val_dice_coeff: 0.7171
Epoch 5/5
200/200 [==============================] - 353s 2s/step - batch: 99.5000 - size: 8.0000 - loss: 3.0359 - rpn_class_loss: 0.0206 - rpn_bbox_loss: 0.7894 - mrcnn_class_loss: 0.4013 - mrcnn_bbox_loss: 0.5923 - mrcnn_mask_loss: 0.4920 - dice_coeff: 0.7403 - val_loss: 3.0742 - val_rpn_class_loss: 0.0213 - val_rpn_bbox_loss: 0.7827 - val_mrcnn_class_loss: 0.4335 - val_mrcnn_bbox_loss: 0.6040 - val_mrcnn_mask_loss: 0.5022 - val_dice_coeff: 0.7305

add_model_results(STATIC_ID)
save_run_state(RUN_ID, STATIC_ID)

plot_loss_graph('MR_2')

plot_loss_graph('MR_3')

plot_loss_graph('MR_4')

add_best_epoch_to_list('MR_2', 'loss', 'min')
add_best_epoch_to_list('MR_3', 'loss', 'min')
add_best_epoch_to_list('MR_4', 'loss', 'min')

Based on the above execution, there is no significant difference depending on the image sizing. Given the performance trends above, we will proceed with stacking a single channel to 3 channels.

Running Training Iterations¶

Iteration 5) Heads/SGD/LR=0.001/Batch=8¶

STATIC_ID='MR_5'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_5(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=723
  VALIDATION_STEPS=128
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_5()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_5_20210606T0751

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_5_20210606T0751
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_5_20210606T0751

EPOCHS=5
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_5_20210606T0751/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
Epoch 1/5
723/723 [==============================] - 642s 867ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.6472 - rpn_class_loss: 0.0250 - rpn_bbox_loss: 0.4550 - mrcnn_class_loss: 0.3250 - mrcnn_bbox_loss: 0.6104 - mrcnn_mask_loss: 0.4730 - dice_coeff: 0.7587 - val_loss: 2.6273 - val_rpn_class_loss: 0.0225 - val_rpn_bbox_loss: 0.4432 - val_mrcnn_class_loss: 0.3426 - val_mrcnn_bbox_loss: 0.5888 - val_mrcnn_mask_loss: 0.4913 - val_dice_coeff: 0.7389
Epoch 2/5
723/723 [==============================] - 579s 802ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5868 - rpn_class_loss: 0.0226 - rpn_bbox_loss: 0.4469 - mrcnn_class_loss: 0.3196 - mrcnn_bbox_loss: 0.5710 - mrcnn_mask_loss: 0.4678 - dice_coeff: 0.7589 - val_loss: 2.6725 - val_rpn_class_loss: 0.0218 - val_rpn_bbox_loss: 0.5188 - val_mrcnn_class_loss: 0.3282 - val_mrcnn_bbox_loss: 0.5770 - val_mrcnn_mask_loss: 0.4579 - val_dice_coeff: 0.7687
Epoch 3/5
723/723 [==============================] - 575s 796ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5784 - rpn_class_loss: 0.0219 - rpn_bbox_loss: 0.4450 - mrcnn_class_loss: 0.3253 - mrcnn_bbox_loss: 0.5606 - mrcnn_mask_loss: 0.4690 - dice_coeff: 0.7566 - val_loss: 2.6477 - val_rpn_class_loss: 0.0212 - val_rpn_bbox_loss: 0.4665 - val_mrcnn_class_loss: 0.3551 - val_mrcnn_bbox_loss: 0.5801 - val_mrcnn_mask_loss: 0.4674 - val_dice_coeff: 0.7573
Epoch 4/5
723/723 [==============================] - 559s 773ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5511 - rpn_class_loss: 0.0214 - rpn_bbox_loss: 0.4313 - mrcnn_class_loss: 0.3222 - mrcnn_bbox_loss: 0.5513 - mrcnn_mask_loss: 0.4654 - dice_coeff: 0.7595 - val_loss: 2.6131 - val_rpn_class_loss: 0.0225 - val_rpn_bbox_loss: 0.4683 - val_mrcnn_class_loss: 0.3256 - val_mrcnn_bbox_loss: 0.5711 - val_mrcnn_mask_loss: 0.4802 - val_dice_coeff: 0.7454
Epoch 5/5
723/723 [==============================] - 538s 744ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5393 - rpn_class_loss: 0.0207 - rpn_bbox_loss: 0.4264 - mrcnn_class_loss: 0.3206 - mrcnn_bbox_loss: 0.5470 - mrcnn_mask_loss: 0.4644 - dice_coeff: 0.7602 - val_loss: 2.7116 - val_rpn_class_loss: 0.0220 - val_rpn_bbox_loss: 0.5286 - val_mrcnn_class_loss: 0.3498 - val_mrcnn_bbox_loss: 0.5853 - val_mrcnn_mask_loss: 0.4699 - val_dice_coeff: 0.7560

add_model_results('MR_5')
add_best_epoch_to_list('MR_5', 'loss', 'min')
save_run_state(RUN_ID, 'MR_5')

plot_loss_graph('MR_5')

Iteration 5.a) Reduce the LR to 0.0005¶

STATIC_ID='MR_5A'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_5a(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.0005
  STEPS_PER_EPOCH=723
  VALIDATION_STEPS=128
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_5a()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_5A_20210606T0927

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_5A_20210606T0927
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_5A_20210606T0927

weights_path = get_checkpoint_weight_path('MR_5', 5)
reset_weights(ID, model, weights_path)

<mrcnn.model.MaskRCNN at 0x7f5290e2b910>

EPOCHS=5
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_5A_20210606T0927/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
Epoch 1/5
723/723 [==============================] - 653s 882ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5847 - rpn_class_loss: 0.0203 - rpn_bbox_loss: 0.4173 - mrcnn_class_loss: 0.3489 - mrcnn_bbox_loss: 0.5646 - mrcnn_mask_loss: 0.4795 - dice_coeff: 0.7541 - val_loss: 2.7722 - val_rpn_class_loss: 0.0235 - val_rpn_bbox_loss: 0.5983 - val_mrcnn_class_loss: 0.3577 - val_mrcnn_bbox_loss: 0.5619 - val_mrcnn_mask_loss: 0.4774 - val_dice_coeff: 0.7535
Epoch 2/5
723/723 [==============================] - 569s 788ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5561 - rpn_class_loss: 0.0200 - rpn_bbox_loss: 0.4133 - mrcnn_class_loss: 0.3438 - mrcnn_bbox_loss: 0.5491 - mrcnn_mask_loss: 0.4756 - dice_coeff: 0.7542 - val_loss: 2.8381 - val_rpn_class_loss: 0.0234 - val_rpn_bbox_loss: 0.6275 - val_mrcnn_class_loss: 0.3783 - val_mrcnn_bbox_loss: 0.5785 - val_mrcnn_mask_loss: 0.4886 - val_dice_coeff: 0.7419
Epoch 3/5
723/723 [==============================] - 533s 738ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5296 - rpn_class_loss: 0.0200 - rpn_bbox_loss: 0.4023 - mrcnn_class_loss: 0.3357 - mrcnn_bbox_loss: 0.5441 - mrcnn_mask_loss: 0.4720 - dice_coeff: 0.7555 - val_loss: 2.9813 - val_rpn_class_loss: 0.0239 - val_rpn_bbox_loss: 0.7901 - val_mrcnn_class_loss: 0.3589 - val_mrcnn_bbox_loss: 0.5801 - val_mrcnn_mask_loss: 0.4764 - val_dice_coeff: 0.7520
Epoch 4/5
723/723 [==============================] - 535s 740ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5215 - rpn_class_loss: 0.0199 - rpn_bbox_loss: 0.4040 - mrcnn_class_loss: 0.3364 - mrcnn_bbox_loss: 0.5354 - mrcnn_mask_loss: 0.4707 - dice_coeff: 0.7551 - val_loss: 2.8322 - val_rpn_class_loss: 0.0230 - val_rpn_bbox_loss: 0.6198 - val_mrcnn_class_loss: 0.3793 - val_mrcnn_bbox_loss: 0.5823 - val_mrcnn_mask_loss: 0.4773 - val_dice_coeff: 0.7504
Epoch 5/5
723/723 [==============================] - 530s 734ms/step - batch: 361.0000 - size: 8.0000 - loss: 2.5078 - rpn_class_loss: 0.0198 - rpn_bbox_loss: 0.3985 - mrcnn_class_loss: 0.3313 - mrcnn_bbox_loss: 0.5326 - mrcnn_mask_loss: 0.4716 - dice_coeff: 0.7538 - val_loss: 2.8417 - val_rpn_class_loss: 0.0241 - val_rpn_bbox_loss: 0.6808 - val_mrcnn_class_loss: 0.3414 - val_mrcnn_bbox_loss: 0.5689 - val_mrcnn_mask_loss: 0.4731 - val_dice_coeff: 0.7534

add_model_results('MR_5A')
add_best_epoch_to_list('MR_5A', 'loss', 'min')
save_run_state(RUN_ID, 'MR_5A')

plot_loss_graph('MR_5A')

Iteration 6) Heads/Lamb/LR=0.001/Batch=8¶

STATIC_ID='MR_6'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_6(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='Lamb'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_6()
ID=get_iteration_id(STATIC_ID)
optimizer = LAMB(learning_rate = mask_config.LEARNING_RATE)

Iteration Id: MR_6_20210606T1201

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/array_ops.py:5049: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.
Instructions for updating:
The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.
Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_6_20210606T1201
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_6_20210606T1201

EPOCHS=5
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_6_20210606T1201/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
Epoch 1/5
200/200 [==============================] - 220s 914ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.7716 - rpn_class_loss: 0.0283 - rpn_bbox_loss: 0.5470 - mrcnn_class_loss: 0.3250 - mrcnn_bbox_loss: 0.6372 - mrcnn_mask_loss: 0.4821 - dice_coeff: 0.7521 - val_loss: 2.6481 - val_rpn_class_loss: 0.0264 - val_rpn_bbox_loss: 0.4199 - val_mrcnn_class_loss: 0.3320 - val_mrcnn_bbox_loss: 0.6394 - val_mrcnn_mask_loss: 0.4722 - val_dice_coeff: 0.7582
Epoch 2/5
200/200 [==============================] - 131s 657ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6245 - rpn_class_loss: 0.0235 - rpn_bbox_loss: 0.4410 - mrcnn_class_loss: 0.3241 - mrcnn_bbox_loss: 0.6065 - mrcnn_mask_loss: 0.4669 - dice_coeff: 0.7624 - val_loss: 2.5957 - val_rpn_class_loss: 0.0259 - val_rpn_bbox_loss: 0.4622 - val_mrcnn_class_loss: 0.3022 - val_mrcnn_bbox_loss: 0.5783 - val_mrcnn_mask_loss: 0.4716 - val_dice_coeff: 0.7556
Epoch 3/5
200/200 [==============================] - 151s 759ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5869 - rpn_class_loss: 0.0221 - rpn_bbox_loss: 0.4341 - mrcnn_class_loss: 0.3165 - mrcnn_bbox_loss: 0.5858 - mrcnn_mask_loss: 0.4682 - dice_coeff: 0.7601 - val_loss: 2.6939 - val_rpn_class_loss: 0.0244 - val_rpn_bbox_loss: 0.5007 - val_mrcnn_class_loss: 0.3510 - val_mrcnn_bbox_loss: 0.5895 - val_mrcnn_mask_loss: 0.4585 - val_dice_coeff: 0.7698
Epoch 4/5
200/200 [==============================] - 149s 749ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6407 - rpn_class_loss: 0.0223 - rpn_bbox_loss: 0.4750 - mrcnn_class_loss: 0.3283 - mrcnn_bbox_loss: 0.5867 - mrcnn_mask_loss: 0.4736 - dice_coeff: 0.7548 - val_loss: 2.8961 - val_rpn_class_loss: 0.0251 - val_rpn_bbox_loss: 0.7307 - val_mrcnn_class_loss: 0.3220 - val_mrcnn_bbox_loss: 0.5901 - val_mrcnn_mask_loss: 0.4526 - val_dice_coeff: 0.7757
Epoch 5/5
200/200 [==============================] - 155s 777ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5954 - rpn_class_loss: 0.0211 - rpn_bbox_loss: 0.4602 - mrcnn_class_loss: 0.3185 - mrcnn_bbox_loss: 0.5692 - mrcnn_mask_loss: 0.4697 - dice_coeff: 0.7567 - val_loss: 2.7717 - val_rpn_class_loss: 0.0238 - val_rpn_bbox_loss: 0.6218 - val_mrcnn_class_loss: 0.3207 - val_mrcnn_bbox_loss: 0.5794 - val_mrcnn_mask_loss: 0.4558 - val_dice_coeff: 0.7702

add_model_results('MR_6')
add_best_epoch_to_list('MR_6', 'loss', 'min')
save_run_state(RUN_ID, 'MR_6')

Iteration 6.a) Train all layers with LR 0.001¶

STATIC_ID='MR_6A'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_6a(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='Lamb'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='all'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_6a()
ID=get_iteration_id(STATIC_ID)
optimizer = LAMB(learning_rate = mask_config.LEARNING_RATE)

Iteration Id: MR_6A_20210606T1216

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_6A_20210606T1216
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_6A_20210606T1216

weights_path = get_checkpoint_weight_path('MR_6', 3)
model=reset_weights(ID, model, weights_path)
model_list[iteration_list[STATIC_ID]]= model

EPOCHS=6
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_6A_20210606T1216/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d_1[0][0]           
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation_40 (Activation)      (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, None, None, 6 0           activation_40[0][0]              
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_41 (Activation)      (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_41[0][0]              
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_42 (Activation)      (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_42[0][0]              
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add_16 (Add)                    (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add_16[0][0]                     
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_43 (Activation)      (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_43[0][0]              
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_44 (Activation)      (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_44[0][0]              
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_17 (Add)                    (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_17[0][0]                     
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_45 (Activation)      (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_45[0][0]              
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_46 (Activation)      (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_46[0][0]              
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_18 (Add)                    (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_18[0][0]                     
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_47 (Activation)      (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_47[0][0]              
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_48 (Activation)      (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_48[0][0]              
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_19 (Add)                    (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_19[0][0]                     
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_49 (Activation)      (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_49[0][0]              
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_50 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_50[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_20 (Add)                    (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_20[0][0]                     
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_51 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_51[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_52 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_52[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_21 (Add)                    (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_21[0][0]                     
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_53 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_53[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_54 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_54[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_22 (Add)                    (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_22[0][0]                     
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_55 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_55[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_56 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_56[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_23 (Add)                    (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_23[0][0]                     
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_57 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_57[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_58 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_58[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_24 (Add)                    (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_24[0][0]                     
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_59 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_59[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_60 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_60[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_25 (Add)                    (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_25[0][0]                     
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_61 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_61[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_62 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_62[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_26 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_26[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_63 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_63[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_64 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_64[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_27 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_27[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_65 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_65[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_66 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_66[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_28 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_28[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_67 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_67[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_68 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_68[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_29 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_29[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_69 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_69[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_70 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_70[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_30 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_30[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_71 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_71[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_72 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_72[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_31 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_31[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda_4[0][0]                   
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_76 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_76[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_77 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_77[0][0]              
__________________________________________________________________________________________________
activation_73 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_73[0][0]              
__________________________________________________________________________________________________
activation_78 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_78[0][0]              
__________________________________________________________________________________________________
activation_74 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_74[0][0]              
__________________________________________________________________________________________________
activation_79 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_79[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_7 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_7[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean_12 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_54 (TensorFlowO [()]                 0           tf_op_layer_Mean_12[0][0]        
__________________________________________________________________________________________________
add_loss_7 (AddLoss)            ()                   0           tf_op_layer_mul_54[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_13 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_55 (TensorFlowO [()]                 0           tf_op_layer_Mean_13[0][0]        
__________________________________________________________________________________________________
add_loss_8 (AddLoss)            ()                   0           tf_op_layer_mul_55[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_14 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_56 (TensorFlowO [()]                 0           tf_op_layer_Mean_14[0][0]        
__________________________________________________________________________________________________
add_loss_9 (AddLoss)            ()                   0           tf_op_layer_mul_56[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_15 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_57 (TensorFlowO [()]                 0           tf_op_layer_Mean_15[0][0]        
__________________________________________________________________________________________________
add_loss_10 (AddLoss)           ()                   0           tf_op_layer_mul_57[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_16 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_58 (TensorFlowO [()]                 0           tf_op_layer_Mean_16[0][0]        
__________________________________________________________________________________________________
add_loss_11 (AddLoss)           ()                   0           tf_op_layer_mul_58[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_17 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_59 (TensorFlowO [()]                 0           tf_op_layer_Mean_17[0][0]        
__________________________________________________________________________________________________
add_loss_12 (AddLoss)           ()                   0           tf_op_layer_mul_59[0][0]         
__________________________________________________________________________________________________
tf_op_layer_AddN_1 (TensorFlowO [()]                 0                                            
__________________________________________________________________________________________________
add_loss_13 (AddLoss)           ()                   0           tf_op_layer_AddN_1[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_18 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_209 (TensorFlow [()]                 0           tf_op_layer_Mean_18[0][0]        
__________________________________________________________________________________________________
add_metric_6 (AddMetric)        ()                   0           tf_op_layer_mul_209[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_19 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_210 (TensorFlow [()]                 0           tf_op_layer_Mean_19[0][0]        
__________________________________________________________________________________________________
add_metric_7 (AddMetric)        ()                   0           tf_op_layer_mul_210[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_20 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_211 (TensorFlow [()]                 0           tf_op_layer_Mean_20[0][0]        
__________________________________________________________________________________________________
add_metric_8 (AddMetric)        ()                   0           tf_op_layer_mul_211[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_21 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_212 (TensorFlow [()]                 0           tf_op_layer_Mean_21[0][0]        
__________________________________________________________________________________________________
add_metric_9 (AddMetric)        ()                   0           tf_op_layer_mul_212[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_22 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_213 (TensorFlow [()]                 0           tf_op_layer_Mean_22[0][0]        
__________________________________________________________________________________________________
add_metric_10 (AddMetric)       ()                   0           tf_op_layer_mul_213[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_23 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_214 (TensorFlow [()]                 0           tf_op_layer_Mean_23[0][0]        
__________________________________________________________________________________________________
add_metric_11 (AddMetric)       ()                   0           tf_op_layer_mul_214[0][0]        
==================================================================================================
Total params: 45,185,182
Trainable params: 45,125,918
Non-trainable params: 59,264
__________________________________________________________________________________________________
Epoch 1/6
200/200 [==============================] - 260s 1s/step - batch: 99.5000 - size: 8.0000 - loss: 2.6306 - rpn_class_loss: 0.0229 - rpn_bbox_loss: 0.5288 - mrcnn_class_loss: 0.2780 - mrcnn_bbox_loss: 0.5699 - mrcnn_mask_loss: 0.5207 - dice_coeff: 0.7102 - val_loss: 2.5384 - val_rpn_class_loss: 0.0252 - val_rpn_bbox_loss: 0.6104 - val_mrcnn_class_loss: 0.1706 - val_mrcnn_bbox_loss: 0.5232 - val_mrcnn_mask_loss: 0.5061 - val_dice_coeff: 0.7029
Epoch 2/6
200/200 [==============================] - 127s 638ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.3984 - rpn_class_loss: 0.0232 - rpn_bbox_loss: 0.5118 - mrcnn_class_loss: 0.1525 - mrcnn_bbox_loss: 0.5072 - mrcnn_mask_loss: 0.4754 - dice_coeff: 0.7282 - val_loss: 2.4025 - val_rpn_class_loss: 0.0257 - val_rpn_bbox_loss: 0.6004 - val_mrcnn_class_loss: 0.0984 - val_mrcnn_bbox_loss: 0.4782 - val_mrcnn_mask_loss: 0.4764 - val_dice_coeff: 0.7234
Epoch 3/6
200/200 [==============================] - 146s 734ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.2705 - rpn_class_loss: 0.0221 - rpn_bbox_loss: 0.5053 - mrcnn_class_loss: 0.0837 - mrcnn_bbox_loss: 0.4701 - mrcnn_mask_loss: 0.4766 - dice_coeff: 0.7127 - val_loss: 2.3449 - val_rpn_class_loss: 0.0221 - val_rpn_bbox_loss: 0.6570 - val_mrcnn_class_loss: 0.0610 - val_mrcnn_bbox_loss: 0.4397 - val_mrcnn_mask_loss: 0.4512 - val_dice_coeff: 0.7139
Epoch 4/6
200/200 [==============================] - 150s 752ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.2403 - rpn_class_loss: 0.0221 - rpn_bbox_loss: 0.5259 - mrcnn_class_loss: 0.0630 - mrcnn_bbox_loss: 0.4658 - mrcnn_mask_loss: 0.4658 - dice_coeff: 0.6978 - val_loss: 2.5030 - val_rpn_class_loss: 0.0244 - val_rpn_bbox_loss: 0.8449 - val_mrcnn_class_loss: 0.0525 - val_mrcnn_bbox_loss: 0.4854 - val_mrcnn_mask_loss: 0.4399 - val_dice_coeff: 0.6559
Epoch 5/6
200/200 [==============================] - 156s 779ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.0666 - rpn_class_loss: 0.0224 - rpn_bbox_loss: 0.5181 - mrcnn_class_loss: 0.0449 - mrcnn_bbox_loss: 0.3981 - mrcnn_mask_loss: 0.4399 - dice_coeff: 0.6432 - val_loss: 2.2001 - val_rpn_class_loss: 0.0236 - val_rpn_bbox_loss: 0.7375 - val_mrcnn_class_loss: 0.0215 - val_mrcnn_bbox_loss: 0.4027 - val_mrcnn_mask_loss: 0.3960 - val_dice_coeff: 0.6187
Epoch 6/6
200/200 [==============================] - 150s 754ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.0689 - rpn_class_loss: 0.0214 - rpn_bbox_loss: 0.5219 - mrcnn_class_loss: 0.0314 - mrcnn_bbox_loss: 0.4076 - mrcnn_mask_loss: 0.4365 - dice_coeff: 0.6501 - val_loss: 2.1019 - val_rpn_class_loss: 0.0227 - val_rpn_bbox_loss: 0.7845 - val_mrcnn_class_loss: 0.0168 - val_mrcnn_bbox_loss: 0.3572 - val_mrcnn_mask_loss: 0.3545 - val_dice_coeff: 0.5663

add_model_results('MR_6A')
add_best_epoch_to_list('MR_6A', 'loss', 'min')
save_run_state(RUN_ID, 'MR_6A')

Iteration 6.b) Train all layers with reduced LR 0.0002¶

STATIC_ID='MR_6B'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_6b(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='Lamb'
  LEARNING_RATE=0.0002
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='all'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_6b()
ID=get_iteration_id(STATIC_ID)
optimizer = LAMB(learning_rate = mask_config.LEARNING_RATE)

Iteration Id: MR_6B_20210606T1239

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/logs/maskrcnn/MR_6B_20210606T1239
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_6B_20210606T1239

weights_path = get_checkpoint_weight_path('MR_6A', 5)
model=reset_weights(ID, model, weights_path)
model_list[iteration_list[STATIC_ID]]= model

EPOCHS=10
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_6B_20210606T1239/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d_2 (ZeroPadding2D (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d_2[0][0]           
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation_80 (Activation)      (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, None, None, 6 0           activation_80[0][0]              
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_81 (Activation)      (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_81[0][0]              
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_82 (Activation)      (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_82[0][0]              
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add_32 (Add)                    (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add_32[0][0]                     
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_83 (Activation)      (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_83[0][0]              
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_84 (Activation)      (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_84[0][0]              
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_33 (Add)                    (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_33[0][0]                     
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_85 (Activation)      (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_85[0][0]              
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_86 (Activation)      (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_86[0][0]              
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_34 (Add)                    (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_34[0][0]                     
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_87 (Activation)      (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_87[0][0]              
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_88 (Activation)      (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_88[0][0]              
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_35 (Add)                    (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_35[0][0]                     
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_89 (Activation)      (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_89[0][0]              
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_90 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_90[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_36 (Add)                    (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_36[0][0]                     
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_91 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_91[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_92 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_92[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_37 (Add)                    (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_37[0][0]                     
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_93 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_93[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_94 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_94[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_38 (Add)                    (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_38[0][0]                     
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_95 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_95[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_96 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_96[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_39 (Add)                    (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_39[0][0]                     
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_97 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_97[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_98 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_98[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_40 (Add)                    (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_40[0][0]                     
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_99 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_99[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_100 (Activation)     (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_100[0][0]             
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_41 (Add)                    (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_41[0][0]                     
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_101 (Activation)     (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_101[0][0]             
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_102 (Activation)     (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_102[0][0]             
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_42 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_42[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_103 (Activation)     (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_103[0][0]             
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_104 (Activation)     (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_104[0][0]             
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_43 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_43[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_105 (Activation)     (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_105[0][0]             
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_106 (Activation)     (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_106[0][0]             
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_44 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_44[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_107 (Activation)     (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_107[0][0]             
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_108 (Activation)     (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_108[0][0]             
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_45 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_45[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_109 (Activation)     (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_109[0][0]             
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_110 (Activation)     (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_110[0][0]             
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_46 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_46[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_111 (Activation)     (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_111[0][0]             
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_112 (Activation)     (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_112[0][0]             
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_47 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_47[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda_8 (Lambda)               (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda_8[0][0]                   
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_116 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_116[0][0]             
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_117 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_117[0][0]             
__________________________________________________________________________________________________
activation_113 (Activation)     (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_113[0][0]             
__________________________________________________________________________________________________
activation_118 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_118[0][0]             
__________________________________________________________________________________________________
activation_114 (Activation)     (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_114[0][0]             
__________________________________________________________________________________________________
activation_119 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_119[0][0]             
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_11 (Lambda)              (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_11[0][0]                  
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean_24 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_215 (TensorFlow [()]                 0           tf_op_layer_Mean_24[0][0]        
__________________________________________________________________________________________________
add_loss_14 (AddLoss)           ()                   0           tf_op_layer_mul_215[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_25 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_216 (TensorFlow [()]                 0           tf_op_layer_Mean_25[0][0]        
__________________________________________________________________________________________________
add_loss_15 (AddLoss)           ()                   0           tf_op_layer_mul_216[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_26 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_217 (TensorFlow [()]                 0           tf_op_layer_Mean_26[0][0]        
__________________________________________________________________________________________________
add_loss_16 (AddLoss)           ()                   0           tf_op_layer_mul_217[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_27 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_218 (TensorFlow [()]                 0           tf_op_layer_Mean_27[0][0]        
__________________________________________________________________________________________________
add_loss_17 (AddLoss)           ()                   0           tf_op_layer_mul_218[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_28 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_219 (TensorFlow [()]                 0           tf_op_layer_Mean_28[0][0]        
__________________________________________________________________________________________________
add_loss_18 (AddLoss)           ()                   0           tf_op_layer_mul_219[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_29 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_220 (TensorFlow [()]                 0           tf_op_layer_Mean_29[0][0]        
__________________________________________________________________________________________________
add_loss_19 (AddLoss)           ()                   0           tf_op_layer_mul_220[0][0]        
__________________________________________________________________________________________________
tf_op_layer_AddN_2 (TensorFlowO [()]                 0                                            
__________________________________________________________________________________________________
add_loss_20 (AddLoss)           ()                   0           tf_op_layer_AddN_2[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_30 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_370 (TensorFlow [()]                 0           tf_op_layer_Mean_30[0][0]        
__________________________________________________________________________________________________
add_metric_12 (AddMetric)       ()                   0           tf_op_layer_mul_370[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_31 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_371 (TensorFlow [()]                 0           tf_op_layer_Mean_31[0][0]        
__________________________________________________________________________________________________
add_metric_13 (AddMetric)       ()                   0           tf_op_layer_mul_371[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_32 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_372 (TensorFlow [()]                 0           tf_op_layer_Mean_32[0][0]        
__________________________________________________________________________________________________
add_metric_14 (AddMetric)       ()                   0           tf_op_layer_mul_372[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_33 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_373 (TensorFlow [()]                 0           tf_op_layer_Mean_33[0][0]        
__________________________________________________________________________________________________
add_metric_15 (AddMetric)       ()                   0           tf_op_layer_mul_373[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_34 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_374 (TensorFlow [()]                 0           tf_op_layer_Mean_34[0][0]        
__________________________________________________________________________________________________
add_metric_16 (AddMetric)       ()                   0           tf_op_layer_mul_374[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_35 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_375 (TensorFlow [()]                 0           tf_op_layer_Mean_35[0][0]        
__________________________________________________________________________________________________
add_metric_17 (AddMetric)       ()                   0           tf_op_layer_mul_375[0][0]        
==================================================================================================
Total params: 45,185,182
Trainable params: 45,125,918
Non-trainable params: 59,264
__________________________________________________________________________________________________
Epoch 1/10
200/200 [==============================] - 287s 1s/step - batch: 99.5000 - size: 8.0000 - loss: 2.2362 - rpn_class_loss: 0.0206 - rpn_bbox_loss: 0.5177 - mrcnn_class_loss: 0.0387 - mrcnn_bbox_loss: 0.4946 - mrcnn_mask_loss: 0.5294 - dice_coeff: 0.6353 - val_loss: 2.1168 - val_rpn_class_loss: 0.0224 - val_rpn_bbox_loss: 0.6287 - val_mrcnn_class_loss: 0.0242 - val_mrcnn_bbox_loss: 0.3639 - val_mrcnn_mask_loss: 0.4431 - val_dice_coeff: 0.6345
Epoch 2/10
200/200 [==============================] - 128s 641ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.0049 - rpn_class_loss: 0.0199 - rpn_bbox_loss: 0.4941 - mrcnn_class_loss: 0.0302 - mrcnn_bbox_loss: 0.3596 - mrcnn_mask_loss: 0.4558 - dice_coeff: 0.6454 - val_loss: 1.9337 - val_rpn_class_loss: 0.0223 - val_rpn_bbox_loss: 0.5968 - val_mrcnn_class_loss: 0.0223 - val_mrcnn_bbox_loss: 0.3373 - val_mrcnn_mask_loss: 0.4071 - val_dice_coeff: 0.5478
Epoch 3/10
200/200 [==============================] - 154s 773ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.9306 - rpn_class_loss: 0.0199 - rpn_bbox_loss: 0.4677 - mrcnn_class_loss: 0.0236 - mrcnn_bbox_loss: 0.3316 - mrcnn_mask_loss: 0.4577 - dice_coeff: 0.6302 - val_loss: 1.9826 - val_rpn_class_loss: 0.0213 - val_rpn_bbox_loss: 0.5275 - val_mrcnn_class_loss: 0.0210 - val_mrcnn_bbox_loss: 0.3289 - val_mrcnn_mask_loss: 0.4535 - val_dice_coeff: 0.6304
Epoch 4/10
200/200 [==============================] - 157s 787ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.9444 - rpn_class_loss: 0.0206 - rpn_bbox_loss: 0.4838 - mrcnn_class_loss: 0.0201 - mrcnn_bbox_loss: 0.3299 - mrcnn_mask_loss: 0.4485 - dice_coeff: 0.6415 - val_loss: 1.9735 - val_rpn_class_loss: 0.0212 - val_rpn_bbox_loss: 0.6555 - val_mrcnn_class_loss: 0.0164 - val_mrcnn_bbox_loss: 0.3100 - val_mrcnn_mask_loss: 0.3989 - val_dice_coeff: 0.5714
Epoch 5/10
200/200 [==============================] - 159s 799ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.8320 - rpn_class_loss: 0.0199 - rpn_bbox_loss: 0.4603 - mrcnn_class_loss: 0.0194 - mrcnn_bbox_loss: 0.3064 - mrcnn_mask_loss: 0.4197 - dice_coeff: 0.6063 - val_loss: 1.9897 - val_rpn_class_loss: 0.0217 - val_rpn_bbox_loss: 0.6010 - val_mrcnn_class_loss: 0.0174 - val_mrcnn_bbox_loss: 0.3289 - val_mrcnn_mask_loss: 0.4181 - val_dice_coeff: 0.6026
Epoch 6/10
200/200 [==============================] - 155s 776ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.7805 - rpn_class_loss: 0.0196 - rpn_bbox_loss: 0.4695 - mrcnn_class_loss: 0.0153 - mrcnn_bbox_loss: 0.2889 - mrcnn_mask_loss: 0.3996 - dice_coeff: 0.5876 - val_loss: 1.8311 - val_rpn_class_loss: 0.0201 - val_rpn_bbox_loss: 0.5782 - val_mrcnn_class_loss: 0.0158 - val_mrcnn_bbox_loss: 0.3012 - val_mrcnn_mask_loss: 0.3780 - val_dice_coeff: 0.5378
Epoch 7/10
200/200 [==============================] - 158s 791ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.7477 - rpn_class_loss: 0.0198 - rpn_bbox_loss: 0.4453 - mrcnn_class_loss: 0.0167 - mrcnn_bbox_loss: 0.2787 - mrcnn_mask_loss: 0.4014 - dice_coeff: 0.5858 - val_loss: 1.9032 - val_rpn_class_loss: 0.0224 - val_rpn_bbox_loss: 0.6889 - val_mrcnn_class_loss: 0.0131 - val_mrcnn_bbox_loss: 0.2562 - val_mrcnn_mask_loss: 0.3640 - val_dice_coeff: 0.5586
Epoch 8/10
200/200 [==============================] - 170s 853ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.8059 - rpn_class_loss: 0.0192 - rpn_bbox_loss: 0.4725 - mrcnn_class_loss: 0.0138 - mrcnn_bbox_loss: 0.2980 - mrcnn_mask_loss: 0.4061 - dice_coeff: 0.5962 - val_loss: 1.9419 - val_rpn_class_loss: 0.0228 - val_rpn_bbox_loss: 0.5965 - val_mrcnn_class_loss: 0.0155 - val_mrcnn_bbox_loss: 0.3094 - val_mrcnn_mask_loss: 0.4308 - val_dice_coeff: 0.5670
Epoch 9/10
200/200 [==============================] - 154s 772ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.6983 - rpn_class_loss: 0.0199 - rpn_bbox_loss: 0.4518 - mrcnn_class_loss: 0.0132 - mrcnn_bbox_loss: 0.2605 - mrcnn_mask_loss: 0.3864 - dice_coeff: 0.5665 - val_loss: 1.7363 - val_rpn_class_loss: 0.0210 - val_rpn_bbox_loss: 0.5831 - val_mrcnn_class_loss: 0.0099 - val_mrcnn_bbox_loss: 0.2532 - val_mrcnn_mask_loss: 0.3474 - val_dice_coeff: 0.5216
Epoch 10/10
200/200 [==============================] - 155s 778ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.6495 - rpn_class_loss: 0.0195 - rpn_bbox_loss: 0.4425 - mrcnn_class_loss: 0.0124 - mrcnn_bbox_loss: 0.2698 - mrcnn_mask_loss: 0.3761 - dice_coeff: 0.5291 - val_loss: 1.8223 - val_rpn_class_loss: 0.0205 - val_rpn_bbox_loss: 0.5800 - val_mrcnn_class_loss: 0.0119 - val_mrcnn_bbox_loss: 0.3042 - val_mrcnn_mask_loss: 0.3734 - val_dice_coeff: 0.5324

add_model_results('MR_6B')
add_best_epoch_to_list('MR_6B', 'loss', 'min')
save_run_state(RUN_ID, 'MR_6B')

Plotting iteration 6 training charts¶

merged_history = pd.concat([model_results[iteration_list['MR_6']], 
                  model_results[iteration_list['MR_6A']],
                  model_results[iteration_list['MR_6B']]],ignore_index=True)

plot_graph(merged_history)

Iteration 7) Heads/SGD/LR=0.001/Batch=8¶

STATIC_ID='MR_7'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_7(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_7()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_7_20210612T0813

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/logs/maskrcnn/MR_7_20210612T0813
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_7_20210612T0813

EPOCHS=5
with tf.device('/device:GPU:0'):
  hist=model.train(dataset_train,
              dataset_val,
              epochs=EPOCHS,
              augmentation=None) 
  add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_7_20210612T0813/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d_1[0][0]           
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation_40 (Activation)      (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, None, None, 6 0           activation_40[0][0]              
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_41 (Activation)      (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_41[0][0]              
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_42 (Activation)      (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_42[0][0]              
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add_16 (Add)                    (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add_16[0][0]                     
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_43 (Activation)      (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_43[0][0]              
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_44 (Activation)      (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_44[0][0]              
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_17 (Add)                    (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_17[0][0]                     
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_45 (Activation)      (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_45[0][0]              
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_46 (Activation)      (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_46[0][0]              
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_18 (Add)                    (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_18[0][0]                     
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_47 (Activation)      (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_47[0][0]              
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_48 (Activation)      (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_48[0][0]              
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_19 (Add)                    (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_19[0][0]                     
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_49 (Activation)      (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_49[0][0]              
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_50 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_50[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_20 (Add)                    (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_20[0][0]                     
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_51 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_51[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_52 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_52[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_21 (Add)                    (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_21[0][0]                     
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_53 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_53[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_54 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_54[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_22 (Add)                    (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_22[0][0]                     
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_55 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_55[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_56 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_56[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_23 (Add)                    (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_23[0][0]                     
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_57 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_57[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_58 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_58[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_24 (Add)                    (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_24[0][0]                     
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_59 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_59[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_60 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_60[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_25 (Add)                    (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_25[0][0]                     
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_61 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_61[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_62 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_62[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_26 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_26[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_63 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_63[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_64 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_64[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_27 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_27[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_65 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_65[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_66 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_66[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_28 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_28[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_67 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_67[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_68 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_68[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_29 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_29[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_69 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_69[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_70 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_70[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_30 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_30[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_71 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_71[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_72 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_72[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_31 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_31[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda_4[0][0]                   
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_76 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_76[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_77 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_77[0][0]              
__________________________________________________________________________________________________
activation_73 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_73[0][0]              
__________________________________________________________________________________________________
activation_78 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_78[0][0]              
__________________________________________________________________________________________________
activation_74 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_74[0][0]              
__________________________________________________________________________________________________
activation_79 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_79[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_7 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_7[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
Epoch 1/5
200/200 [==============================] - 158s 622ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.7806 - rpn_class_loss: 0.0286 - rpn_bbox_loss: 0.4780 - mrcnn_class_loss: 0.3991 - mrcnn_bbox_loss: 0.6399 - mrcnn_mask_loss: 0.4920 - dice_coeff: 0.7431 - val_loss: 2.5708 - val_rpn_class_loss: 0.0257 - val_rpn_bbox_loss: 0.4051 - val_mrcnn_class_loss: 0.3137 - val_mrcnn_bbox_loss: 0.5893 - val_mrcnn_mask_loss: 0.5200 - val_dice_coeff: 0.7171
Epoch 2/5
200/200 [==============================] - 83s 418ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6182 - rpn_class_loss: 0.0223 - rpn_bbox_loss: 0.4246 - mrcnn_class_loss: 0.3377 - mrcnn_bbox_loss: 0.6045 - mrcnn_mask_loss: 0.4599 - dice_coeff: 0.7690 - val_loss: 2.7184 - val_rpn_class_loss: 0.0250 - val_rpn_bbox_loss: 0.4279 - val_mrcnn_class_loss: 0.4353 - val_mrcnn_bbox_loss: 0.6014 - val_mrcnn_mask_loss: 0.4612 - val_dice_coeff: 0.7676
Epoch 3/5
200/200 [==============================] - 103s 517ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6602 - rpn_class_loss: 0.0219 - rpn_bbox_loss: 0.4479 - mrcnn_class_loss: 0.3582 - mrcnn_bbox_loss: 0.6032 - mrcnn_mask_loss: 0.4687 - dice_coeff: 0.7603 - val_loss: 2.7272 - val_rpn_class_loss: 0.0248 - val_rpn_bbox_loss: 0.5403 - val_mrcnn_class_loss: 0.3434 - val_mrcnn_bbox_loss: 0.5903 - val_mrcnn_mask_loss: 0.4594 - val_dice_coeff: 0.7690
Epoch 4/5
200/200 [==============================] - 103s 515ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6505 - rpn_class_loss: 0.0223 - rpn_bbox_loss: 0.4643 - mrcnn_class_loss: 0.3519 - mrcnn_bbox_loss: 0.5853 - mrcnn_mask_loss: 0.4660 - dice_coeff: 0.7608 - val_loss: 2.7778 - val_rpn_class_loss: 0.0233 - val_rpn_bbox_loss: 0.6083 - val_mrcnn_class_loss: 0.3519 - val_mrcnn_bbox_loss: 0.5700 - val_mrcnn_mask_loss: 0.4450 - val_dice_coeff: 0.7793
Epoch 5/5
200/200 [==============================] - 104s 524ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5976 - rpn_class_loss: 0.0212 - rpn_bbox_loss: 0.4408 - mrcnn_class_loss: 0.3428 - mrcnn_bbox_loss: 0.5673 - mrcnn_mask_loss: 0.4655 - dice_coeff: 0.7600 - val_loss: 2.7031 - val_rpn_class_loss: 0.0232 - val_rpn_bbox_loss: 0.5524 - val_mrcnn_class_loss: 0.3447 - val_mrcnn_bbox_loss: 0.5595 - val_mrcnn_mask_loss: 0.4580 - val_dice_coeff: 0.7653

add_model_results('MR_7')
add_best_epoch_to_list('MR_7', 'loss', 'min')
save_run_state(RUN_ID, 'MR_7')

Iteration 7.a) Train all layers with LR 0.001¶

STATIC_ID='MR_7A'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_7a(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='all'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_7a()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_7A_20210612T0828

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/logs/maskrcnn/MR_7A_20210612T0828
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_7A_20210612T0828

weights_path = get_checkpoint_weight_path('MR_7', 5)
model=reset_weights(ID, model, weights_path)
model_list[iteration_list[STATIC_ID]]= model

EPOCHS=6
with tf.device('/device:GPU:0'):
  hist=model.train(dataset_train,
              dataset_val,
              epochs=EPOCHS,
              augmentation=None) 
  add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_7A_20210612T0828/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d_2 (ZeroPadding2D (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d_2[0][0]           
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation_80 (Activation)      (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)  (None, None, None, 6 0           activation_80[0][0]              
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_81 (Activation)      (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_81[0][0]              
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_82 (Activation)      (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_82[0][0]              
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d_2[0][0]            
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add_32 (Add)                    (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add_32[0][0]                     
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_83 (Activation)      (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_83[0][0]              
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_84 (Activation)      (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_84[0][0]              
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_33 (Add)                    (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_33[0][0]                     
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_85 (Activation)      (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_85[0][0]              
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_86 (Activation)      (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_86[0][0]              
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_34 (Add)                    (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_34[0][0]                     
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_87 (Activation)      (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_87[0][0]              
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_88 (Activation)      (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_88[0][0]              
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_35 (Add)                    (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_35[0][0]                     
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_89 (Activation)      (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_89[0][0]              
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_90 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_90[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_36 (Add)                    (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_36[0][0]                     
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_91 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_91[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_92 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_92[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_37 (Add)                    (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_37[0][0]                     
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_93 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_93[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_94 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_94[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_38 (Add)                    (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_38[0][0]                     
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_95 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_95[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_96 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_96[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_39 (Add)                    (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_39[0][0]                     
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_97 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_97[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_98 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_98[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_40 (Add)                    (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_40[0][0]                     
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_99 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_99[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_100 (Activation)     (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_100[0][0]             
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_41 (Add)                    (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_41[0][0]                     
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_101 (Activation)     (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_101[0][0]             
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_102 (Activation)     (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_102[0][0]             
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_42 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_42[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_103 (Activation)     (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_103[0][0]             
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_104 (Activation)     (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_104[0][0]             
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_43 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_43[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_105 (Activation)     (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_105[0][0]             
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_106 (Activation)     (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_106[0][0]             
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_44 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_44[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_107 (Activation)     (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_107[0][0]             
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_108 (Activation)     (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_108[0][0]             
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_45 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_45[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_109 (Activation)     (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_109[0][0]             
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_110 (Activation)     (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_110[0][0]             
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_46 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_46[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_111 (Activation)     (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_111[0][0]             
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_112 (Activation)     (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_112[0][0]             
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_47 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_47[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda_8 (Lambda)               (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda_8[0][0]                   
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_116 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_116[0][0]             
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_117 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_117[0][0]             
__________________________________________________________________________________________________
activation_113 (Activation)     (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_113[0][0]             
__________________________________________________________________________________________________
activation_118 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_118[0][0]             
__________________________________________________________________________________________________
activation_114 (Activation)     (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_114[0][0]             
__________________________________________________________________________________________________
activation_119 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_119[0][0]             
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_11 (Lambda)              (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_11[0][0]                  
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean_12 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_54 (TensorFlowO [()]                 0           tf_op_layer_Mean_12[0][0]        
__________________________________________________________________________________________________
add_loss_7 (AddLoss)            ()                   0           tf_op_layer_mul_54[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_13 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_55 (TensorFlowO [()]                 0           tf_op_layer_Mean_13[0][0]        
__________________________________________________________________________________________________
add_loss_8 (AddLoss)            ()                   0           tf_op_layer_mul_55[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_14 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_56 (TensorFlowO [()]                 0           tf_op_layer_Mean_14[0][0]        
__________________________________________________________________________________________________
add_loss_9 (AddLoss)            ()                   0           tf_op_layer_mul_56[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_15 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_57 (TensorFlowO [()]                 0           tf_op_layer_Mean_15[0][0]        
__________________________________________________________________________________________________
add_loss_10 (AddLoss)           ()                   0           tf_op_layer_mul_57[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_16 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_58 (TensorFlowO [()]                 0           tf_op_layer_Mean_16[0][0]        
__________________________________________________________________________________________________
add_loss_11 (AddLoss)           ()                   0           tf_op_layer_mul_58[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_17 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_59 (TensorFlowO [()]                 0           tf_op_layer_Mean_17[0][0]        
__________________________________________________________________________________________________
add_loss_12 (AddLoss)           ()                   0           tf_op_layer_mul_59[0][0]         
__________________________________________________________________________________________________
tf_op_layer_AddN_1 (TensorFlowO [()]                 0                                            
__________________________________________________________________________________________________
add_loss_13 (AddLoss)           ()                   0           tf_op_layer_AddN_1[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_18 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_209 (TensorFlow [()]                 0           tf_op_layer_Mean_18[0][0]        
__________________________________________________________________________________________________
add_metric_6 (AddMetric)        ()                   0           tf_op_layer_mul_209[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_19 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_210 (TensorFlow [()]                 0           tf_op_layer_Mean_19[0][0]        
__________________________________________________________________________________________________
add_metric_7 (AddMetric)        ()                   0           tf_op_layer_mul_210[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_20 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_211 (TensorFlow [()]                 0           tf_op_layer_Mean_20[0][0]        
__________________________________________________________________________________________________
add_metric_8 (AddMetric)        ()                   0           tf_op_layer_mul_211[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_21 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_212 (TensorFlow [()]                 0           tf_op_layer_Mean_21[0][0]        
__________________________________________________________________________________________________
add_metric_9 (AddMetric)        ()                   0           tf_op_layer_mul_212[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_22 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_213 (TensorFlow [()]                 0           tf_op_layer_Mean_22[0][0]        
__________________________________________________________________________________________________
add_metric_10 (AddMetric)       ()                   0           tf_op_layer_mul_213[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_23 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_214 (TensorFlow [()]                 0           tf_op_layer_Mean_23[0][0]        
__________________________________________________________________________________________________
add_metric_11 (AddMetric)       ()                   0           tf_op_layer_mul_214[0][0]        
==================================================================================================
Total params: 45,185,182
Trainable params: 45,125,918
Non-trainable params: 59,264
__________________________________________________________________________________________________
Epoch 1/6
200/200 [==============================] - 177s 749ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6561 - rpn_class_loss: 0.0217 - rpn_bbox_loss: 0.4572 - mrcnn_class_loss: 0.3527 - mrcnn_bbox_loss: 0.5868 - mrcnn_mask_loss: 0.5004 - dice_coeff: 0.7372 - val_loss: 2.6781 - val_rpn_class_loss: 0.0225 - val_rpn_bbox_loss: 0.4775 - val_mrcnn_class_loss: 0.3730 - val_mrcnn_bbox_loss: 0.5768 - val_mrcnn_mask_loss: 0.4689 - val_dice_coeff: 0.7593
Epoch 2/6
200/200 [==============================] - 83s 416ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6269 - rpn_class_loss: 0.0199 - rpn_bbox_loss: 0.4430 - mrcnn_class_loss: 0.3644 - mrcnn_bbox_loss: 0.5739 - mrcnn_mask_loss: 0.4694 - dice_coeff: 0.7563 - val_loss: 2.6843 - val_rpn_class_loss: 0.0225 - val_rpn_bbox_loss: 0.5207 - val_mrcnn_class_loss: 0.3490 - val_mrcnn_bbox_loss: 0.5679 - val_mrcnn_mask_loss: 0.4704 - val_dice_coeff: 0.7538
Epoch 3/6
200/200 [==============================] - 105s 524ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6124 - rpn_class_loss: 0.0197 - rpn_bbox_loss: 0.4400 - mrcnn_class_loss: 0.3583 - mrcnn_bbox_loss: 0.5690 - mrcnn_mask_loss: 0.4699 - dice_coeff: 0.7555 - val_loss: 2.7101 - val_rpn_class_loss: 0.0223 - val_rpn_bbox_loss: 0.5678 - val_mrcnn_class_loss: 0.3239 - val_mrcnn_bbox_loss: 0.5701 - val_mrcnn_mask_loss: 0.4549 - val_dice_coeff: 0.7712
Epoch 4/6
200/200 [==============================] - 102s 509ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6161 - rpn_class_loss: 0.0203 - rpn_bbox_loss: 0.4520 - mrcnn_class_loss: 0.3525 - mrcnn_bbox_loss: 0.5665 - mrcnn_mask_loss: 0.4657 - dice_coeff: 0.7591 - val_loss: 2.7936 - val_rpn_class_loss: 0.0215 - val_rpn_bbox_loss: 0.6056 - val_mrcnn_class_loss: 0.3740 - val_mrcnn_bbox_loss: 0.5685 - val_mrcnn_mask_loss: 0.4494 - val_dice_coeff: 0.7746
Epoch 5/6
200/200 [==============================] - 104s 521ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5892 - rpn_class_loss: 0.0195 - rpn_bbox_loss: 0.4395 - mrcnn_class_loss: 0.3514 - mrcnn_bbox_loss: 0.5556 - mrcnn_mask_loss: 0.4640 - dice_coeff: 0.7592 - val_loss: 2.7607 - val_rpn_class_loss: 0.0211 - val_rpn_bbox_loss: 0.5843 - val_mrcnn_class_loss: 0.3702 - val_mrcnn_bbox_loss: 0.5628 - val_mrcnn_mask_loss: 0.4579 - val_dice_coeff: 0.7645
Epoch 6/6
200/200 [==============================] - 104s 521ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6121 - rpn_class_loss: 0.0195 - rpn_bbox_loss: 0.4456 - mrcnn_class_loss: 0.3615 - mrcnn_bbox_loss: 0.5620 - mrcnn_mask_loss: 0.4628 - dice_coeff: 0.7607 - val_loss: 2.6259 - val_rpn_class_loss: 0.0200 - val_rpn_bbox_loss: 0.5179 - val_mrcnn_class_loss: 0.3100 - val_mrcnn_bbox_loss: 0.5547 - val_mrcnn_mask_loss: 0.4640 - val_dice_coeff: 0.7594

add_model_results('MR_7A')
add_best_epoch_to_list('MR_7A', 'loss', 'min')
save_run_state(RUN_ID, 'MR_7A')

Iteration 7.b) Train all layers with reduced LR 0.0002¶

STATIC_ID='MR_7B'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_7b(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='SGD'
  LEARNING_RATE=0.0002
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'dice_coeff': 1.0}
  LAYERS='all'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_7b()
ID=get_iteration_id(STATIC_ID)
optimizer = tf.keras.optimizers.SGD(lr=mask_config.LEARNING_RATE, momentum=mask_config.LEARNING_MOMENTUM, clipnorm=mask_config.GRADIENT_CLIP_NORM)

Iteration Id: MR_7B_20210612T0850

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/logs/maskrcnn/MR_7B_20210612T0850
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_7B_20210612T0850

weights_path = get_checkpoint_weight_path('MR_7A', 5)
model=reset_weights(ID, model, weights_path)
model_list[iteration_list[STATIC_ID]]= model

EPOCHS=10
with tf.device('/device:GPU:0'):
  hist=model.train(dataset_train,
              dataset_val,
              epochs=EPOCHS,
              augmentation=None) 
  add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_7B_20210612T0850/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d_3 (ZeroPadding2D (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d_3[0][0]           
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation_120 (Activation)     (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D)  (None, None, None, 6 0           activation_120[0][0]             
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d_3[0][0]            
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_121 (Activation)     (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_121[0][0]             
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_122 (Activation)     (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_122[0][0]             
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d_3[0][0]            
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add_48 (Add)                    (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add_48[0][0]                     
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_123 (Activation)     (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_123[0][0]             
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_124 (Activation)     (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_124[0][0]             
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_49 (Add)                    (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_49[0][0]                     
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_125 (Activation)     (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_125[0][0]             
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_126 (Activation)     (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_126[0][0]             
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_50 (Add)                    (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_50[0][0]                     
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_127 (Activation)     (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_127[0][0]             
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_128 (Activation)     (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_128[0][0]             
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_51 (Add)                    (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_51[0][0]                     
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_129 (Activation)     (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_129[0][0]             
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_130 (Activation)     (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_130[0][0]             
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_52 (Add)                    (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_52[0][0]                     
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_131 (Activation)     (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_131[0][0]             
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_132 (Activation)     (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_132[0][0]             
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_53 (Add)                    (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_53[0][0]                     
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_133 (Activation)     (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_133[0][0]             
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_134 (Activation)     (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_134[0][0]             
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_54 (Add)                    (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_54[0][0]                     
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_135 (Activation)     (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_135[0][0]             
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_136 (Activation)     (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_136[0][0]             
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_55 (Add)                    (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_55[0][0]                     
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_137 (Activation)     (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_137[0][0]             
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_138 (Activation)     (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_138[0][0]             
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_56 (Add)                    (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_56[0][0]                     
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_139 (Activation)     (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_139[0][0]             
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_140 (Activation)     (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_140[0][0]             
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_57 (Add)                    (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_57[0][0]                     
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_141 (Activation)     (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_141[0][0]             
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_142 (Activation)     (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_142[0][0]             
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_58 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_58[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_143 (Activation)     (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_143[0][0]             
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_144 (Activation)     (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_144[0][0]             
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_59 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_59[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_145 (Activation)     (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_145[0][0]             
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_146 (Activation)     (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_146[0][0]             
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_60 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_60[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_147 (Activation)     (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_147[0][0]             
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_148 (Activation)     (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_148[0][0]             
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_61 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_61[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_149 (Activation)     (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_149[0][0]             
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_150 (Activation)     (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_150[0][0]             
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_62 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_62[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_151 (Activation)     (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_151[0][0]             
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_152 (Activation)     (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_152[0][0]             
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_63 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_63[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda_12 (Lambda)              (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda_12[0][0]                  
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_156 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_156[0][0]             
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_157 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_157[0][0]             
__________________________________________________________________________________________________
activation_153 (Activation)     (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_153[0][0]             
__________________________________________________________________________________________________
activation_158 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_158[0][0]             
__________________________________________________________________________________________________
activation_154 (Activation)     (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_154[0][0]             
__________________________________________________________________________________________________
activation_159 (Activation)     (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_159[0][0]             
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_15 (Lambda)              (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_15[0][0]                  
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean_24 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_215 (TensorFlow [()]                 0           tf_op_layer_Mean_24[0][0]        
__________________________________________________________________________________________________
add_loss_14 (AddLoss)           ()                   0           tf_op_layer_mul_215[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_25 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_216 (TensorFlow [()]                 0           tf_op_layer_Mean_25[0][0]        
__________________________________________________________________________________________________
add_loss_15 (AddLoss)           ()                   0           tf_op_layer_mul_216[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_26 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_217 (TensorFlow [()]                 0           tf_op_layer_Mean_26[0][0]        
__________________________________________________________________________________________________
add_loss_16 (AddLoss)           ()                   0           tf_op_layer_mul_217[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_27 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_218 (TensorFlow [()]                 0           tf_op_layer_Mean_27[0][0]        
__________________________________________________________________________________________________
add_loss_17 (AddLoss)           ()                   0           tf_op_layer_mul_218[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_28 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_219 (TensorFlow [()]                 0           tf_op_layer_Mean_28[0][0]        
__________________________________________________________________________________________________
add_loss_18 (AddLoss)           ()                   0           tf_op_layer_mul_219[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_29 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_220 (TensorFlow [()]                 0           tf_op_layer_Mean_29[0][0]        
__________________________________________________________________________________________________
add_loss_19 (AddLoss)           ()                   0           tf_op_layer_mul_220[0][0]        
__________________________________________________________________________________________________
tf_op_layer_AddN_2 (TensorFlowO [()]                 0                                            
__________________________________________________________________________________________________
add_loss_20 (AddLoss)           ()                   0           tf_op_layer_AddN_2[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_30 (TensorFlow [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_370 (TensorFlow [()]                 0           tf_op_layer_Mean_30[0][0]        
__________________________________________________________________________________________________
add_metric_12 (AddMetric)       ()                   0           tf_op_layer_mul_370[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_31 (TensorFlow [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_371 (TensorFlow [()]                 0           tf_op_layer_Mean_31[0][0]        
__________________________________________________________________________________________________
add_metric_13 (AddMetric)       ()                   0           tf_op_layer_mul_371[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_32 (TensorFlow [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_372 (TensorFlow [()]                 0           tf_op_layer_Mean_32[0][0]        
__________________________________________________________________________________________________
add_metric_14 (AddMetric)       ()                   0           tf_op_layer_mul_372[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_33 (TensorFlow [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_373 (TensorFlow [()]                 0           tf_op_layer_Mean_33[0][0]        
__________________________________________________________________________________________________
add_metric_15 (AddMetric)       ()                   0           tf_op_layer_mul_373[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_34 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_374 (TensorFlow [()]                 0           tf_op_layer_Mean_34[0][0]        
__________________________________________________________________________________________________
add_metric_16 (AddMetric)       ()                   0           tf_op_layer_mul_374[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_35 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_375 (TensorFlow [()]                 0           tf_op_layer_Mean_35[0][0]        
__________________________________________________________________________________________________
add_metric_17 (AddMetric)       ()                   0           tf_op_layer_mul_375[0][0]        
==================================================================================================
Total params: 45,185,182
Trainable params: 45,125,918
Non-trainable params: 59,264
__________________________________________________________________________________________________
Epoch 1/10
200/200 [==============================] - 193s 810ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6271 - rpn_class_loss: 0.0191 - rpn_bbox_loss: 0.4423 - mrcnn_class_loss: 0.3726 - mrcnn_bbox_loss: 0.5614 - mrcnn_mask_loss: 0.4922 - dice_coeff: 0.7395 - val_loss: 2.6534 - val_rpn_class_loss: 0.0211 - val_rpn_bbox_loss: 0.5000 - val_mrcnn_class_loss: 0.3772 - val_mrcnn_bbox_loss: 0.5333 - val_mrcnn_mask_loss: 0.4671 - val_dice_coeff: 0.7547
Epoch 2/10
200/200 [==============================] - 85s 426ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6017 - rpn_class_loss: 0.0189 - rpn_bbox_loss: 0.4301 - mrcnn_class_loss: 0.3803 - mrcnn_bbox_loss: 0.5500 - mrcnn_mask_loss: 0.4658 - dice_coeff: 0.7566 - val_loss: 2.7015 - val_rpn_class_loss: 0.0217 - val_rpn_bbox_loss: 0.5308 - val_mrcnn_class_loss: 0.3679 - val_mrcnn_bbox_loss: 0.5578 - val_mrcnn_mask_loss: 0.4643 - val_dice_coeff: 0.7589
Epoch 3/10
200/200 [==============================] - 103s 518ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5859 - rpn_class_loss: 0.0191 - rpn_bbox_loss: 0.4230 - mrcnn_class_loss: 0.3760 - mrcnn_bbox_loss: 0.5453 - mrcnn_mask_loss: 0.4675 - dice_coeff: 0.7550 - val_loss: 2.6391 - val_rpn_class_loss: 0.0206 - val_rpn_bbox_loss: 0.4791 - val_mrcnn_class_loss: 0.3710 - val_mrcnn_bbox_loss: 0.5473 - val_mrcnn_mask_loss: 0.4694 - val_dice_coeff: 0.7516
Epoch 4/10
200/200 [==============================] - 98s 492ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5974 - rpn_class_loss: 0.0197 - rpn_bbox_loss: 0.4378 - mrcnn_class_loss: 0.3723 - mrcnn_bbox_loss: 0.5447 - mrcnn_mask_loss: 0.4661 - dice_coeff: 0.7567 - val_loss: 2.7549 - val_rpn_class_loss: 0.0210 - val_rpn_bbox_loss: 0.5668 - val_mrcnn_class_loss: 0.3859 - val_mrcnn_bbox_loss: 0.5580 - val_mrcnn_mask_loss: 0.4633 - val_dice_coeff: 0.7598
Epoch 5/10
200/200 [==============================] - 104s 521ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5771 - rpn_class_loss: 0.0190 - rpn_bbox_loss: 0.4235 - mrcnn_class_loss: 0.3730 - mrcnn_bbox_loss: 0.5402 - mrcnn_mask_loss: 0.4642 - dice_coeff: 0.7572 - val_loss: 2.7214 - val_rpn_class_loss: 0.0210 - val_rpn_bbox_loss: 0.5412 - val_mrcnn_class_loss: 0.3741 - val_mrcnn_bbox_loss: 0.5627 - val_mrcnn_mask_loss: 0.4659 - val_dice_coeff: 0.7565
Epoch 6/10
200/200 [==============================] - 100s 499ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5900 - rpn_class_loss: 0.0191 - rpn_bbox_loss: 0.4311 - mrcnn_class_loss: 0.3710 - mrcnn_bbox_loss: 0.5468 - mrcnn_mask_loss: 0.4640 - dice_coeff: 0.7581 - val_loss: 2.6744 - val_rpn_class_loss: 0.0199 - val_rpn_bbox_loss: 0.5340 - val_mrcnn_class_loss: 0.3454 - val_mrcnn_bbox_loss: 0.5530 - val_mrcnn_mask_loss: 0.4579 - val_dice_coeff: 0.7641
Epoch 7/10
200/200 [==============================] - 102s 511ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5676 - rpn_class_loss: 0.0193 - rpn_bbox_loss: 0.4226 - mrcnn_class_loss: 0.3586 - mrcnn_bbox_loss: 0.5450 - mrcnn_mask_loss: 0.4667 - dice_coeff: 0.7554 - val_loss: 2.7432 - val_rpn_class_loss: 0.0216 - val_rpn_bbox_loss: 0.5945 - val_mrcnn_class_loss: 0.3544 - val_mrcnn_bbox_loss: 0.5514 - val_mrcnn_mask_loss: 0.4667 - val_dice_coeff: 0.7545
Epoch 8/10
200/200 [==============================] - 109s 546ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.6011 - rpn_class_loss: 0.0187 - rpn_bbox_loss: 0.4444 - mrcnn_class_loss: 0.3685 - mrcnn_bbox_loss: 0.5479 - mrcnn_mask_loss: 0.4624 - dice_coeff: 0.7591 - val_loss: 2.6967 - val_rpn_class_loss: 0.0218 - val_rpn_bbox_loss: 0.5430 - val_mrcnn_class_loss: 0.3474 - val_mrcnn_bbox_loss: 0.5612 - val_mrcnn_mask_loss: 0.4678 - val_dice_coeff: 0.7555
Epoch 9/10
200/200 [==============================] - 103s 515ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5725 - rpn_class_loss: 0.0195 - rpn_bbox_loss: 0.4293 - mrcnn_class_loss: 0.3596 - mrcnn_bbox_loss: 0.5429 - mrcnn_mask_loss: 0.4650 - dice_coeff: 0.7562 - val_loss: 2.7137 - val_rpn_class_loss: 0.0209 - val_rpn_bbox_loss: 0.5244 - val_mrcnn_class_loss: 0.3883 - val_mrcnn_bbox_loss: 0.5575 - val_mrcnn_mask_loss: 0.4667 - val_dice_coeff: 0.7559
Epoch 10/10
200/200 [==============================] - 104s 521ms/step - batch: 99.5000 - size: 8.0000 - loss: 2.5759 - rpn_class_loss: 0.0191 - rpn_bbox_loss: 0.4306 - mrcnn_class_loss: 0.3610 - mrcnn_bbox_loss: 0.5433 - mrcnn_mask_loss: 0.4656 - dice_coeff: 0.7563 - val_loss: 2.7003 - val_rpn_class_loss: 0.0203 - val_rpn_bbox_loss: 0.5498 - val_mrcnn_class_loss: 0.3613 - val_mrcnn_bbox_loss: 0.5477 - val_mrcnn_mask_loss: 0.4654 - val_dice_coeff: 0.7558

add_model_results('MR_7B')
add_best_epoch_to_list('MR_7B', 'loss', 'min')
save_run_state(RUN_ID, 'MR_7B')

Plotting itertion 7 training charts¶

merged_history = pd.concat([model_results[iteration_list['MR_7']], 
                  model_results[iteration_list['MR_7A']],
                  model_results[iteration_list['MR_7B']]],ignore_index=True)

plot_graph(merged_history)

Iteration 8) Heads/Lamb/LR=0.001/Batch=8¶

STATIC_ID='MR_8'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_8(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='Lamb'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 0., 'dice_coeff': 0.}
  LAYERS='heads'

# Config time settings - Cannot be changed at runtime.
# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_8()
ID=get_iteration_id(STATIC_ID)
optimizer = LAMB(learning_rate = mask_config.LEARNING_RATE)

Iteration Id: MR_8_20210612T1234

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/logs/maskrcnn/MR_8_20210612T1234
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_8_20210612T1234

EPOCHS=5
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_8_20210612T1234/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_48 (TensorFlowO [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_48[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_49 (TensorFlowO [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_49[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_50 (TensorFlowO [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_50[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_51 (TensorFlowO [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_51[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_52 (TensorFlowO [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_52[0][0]         
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_53 (TensorFlowO [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_53[0][0]         
==================================================================================================
Total params: 45,185,182
Trainable params: 21,069,086
Non-trainable params: 24,116,096
__________________________________________________________________________________________________
Epoch 1/5
200/200 [==============================] - 153s 666ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.5211 - rpn_class_loss: 0.0321 - rpn_bbox_loss: 0.5310 - mrcnn_class_loss: 0.3263 - mrcnn_bbox_loss: 0.6317 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.4338 - val_rpn_class_loss: 0.0263 - val_rpn_bbox_loss: 0.4925 - val_mrcnn_class_loss: 0.3243 - val_mrcnn_bbox_loss: 0.5906 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 2/5
200/200 [==============================] - 81s 408ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.4254 - rpn_class_loss: 0.0220 - rpn_bbox_loss: 0.4567 - mrcnn_class_loss: 0.3507 - mrcnn_bbox_loss: 0.5960 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.5202 - val_rpn_class_loss: 0.0255 - val_rpn_bbox_loss: 0.5740 - val_mrcnn_class_loss: 0.3527 - val_mrcnn_bbox_loss: 0.5680 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 3/5
200/200 [==============================] - 101s 506ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.3994 - rpn_class_loss: 0.0212 - rpn_bbox_loss: 0.4552 - mrcnn_class_loss: 0.3430 - mrcnn_bbox_loss: 0.5800 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.5026 - val_rpn_class_loss: 0.0231 - val_rpn_bbox_loss: 0.5271 - val_mrcnn_class_loss: 0.3714 - val_mrcnn_bbox_loss: 0.5810 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 4/5
200/200 [==============================] - 103s 517ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.4211 - rpn_class_loss: 0.0209 - rpn_bbox_loss: 0.4571 - mrcnn_class_loss: 0.3705 - mrcnn_bbox_loss: 0.5725 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.5492 - val_rpn_class_loss: 0.0229 - val_rpn_bbox_loss: 0.6116 - val_mrcnn_class_loss: 0.3512 - val_mrcnn_bbox_loss: 0.5635 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 5/5
200/200 [==============================] - 103s 516ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.3754 - rpn_class_loss: 0.0200 - rpn_bbox_loss: 0.4381 - mrcnn_class_loss: 0.3537 - mrcnn_bbox_loss: 0.5636 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.5322 - val_rpn_class_loss: 0.0226 - val_rpn_bbox_loss: 0.5645 - val_mrcnn_class_loss: 0.3823 - val_mrcnn_bbox_loss: 0.5628 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00

add_model_results('MR_8')
add_best_epoch_to_list('MR_8', 'loss', 'min')
save_run_state(RUN_ID, 'MR_8')

Iteration 8.a) Train all layers with LR 0.001¶

K.clear_session()
STATIC_ID='MR_8A'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_8a(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='Lamb'
  LEARNING_RATE=0.001
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 0., 'dice_coeff': 0.}
  LAYERS='all'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_8a()
ID=get_iteration_id(STATIC_ID)
optimizer = LAMB(learning_rate = mask_config.LEARNING_RATE)

Iteration Id: MR_8A_20210612T1249

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/logs/maskrcnn/MR_8A_20210612T1249
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_8A_20210612T1249

weights_path = get_checkpoint_weight_path('MR_8', 5)
model=reset_weights(ID, model, weights_path)
model_list[iteration_list[STATIC_ID]]= model

EPOCHS=6
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_8A_20210612T1249/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_155 (TensorFlow [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_155[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_156 (TensorFlow [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_156[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_157 (TensorFlow [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_157[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_158 (TensorFlow [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_158[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_159 (TensorFlow [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_159[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_160 (TensorFlow [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_160[0][0]        
==================================================================================================
Total params: 45,185,182
Trainable params: 45,125,918
Non-trainable params: 59,264
__________________________________________________________________________________________________
Epoch 1/6
200/200 [==============================] - 181s 659ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.3658 - rpn_class_loss: 0.0220 - rpn_bbox_loss: 0.4796 - mrcnn_class_loss: 0.2986 - mrcnn_bbox_loss: 0.5657 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.2535 - val_rpn_class_loss: 0.0243 - val_rpn_bbox_loss: 0.5113 - val_mrcnn_class_loss: 0.1885 - val_mrcnn_bbox_loss: 0.5295 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 2/6
200/200 [==============================] - 87s 438ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.1928 - rpn_class_loss: 0.0214 - rpn_bbox_loss: 0.5050 - mrcnn_class_loss: 0.1654 - mrcnn_bbox_loss: 0.5010 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.3512 - val_rpn_class_loss: 0.0284 - val_rpn_bbox_loss: 0.7652 - val_mrcnn_class_loss: 0.0723 - val_mrcnn_bbox_loss: 0.4854 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 3/6
200/200 [==============================] - 102s 511ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.0982 - rpn_class_loss: 0.0221 - rpn_bbox_loss: 0.5187 - mrcnn_class_loss: 0.0797 - mrcnn_bbox_loss: 0.4777 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.1681 - val_rpn_class_loss: 0.0231 - val_rpn_bbox_loss: 0.6252 - val_mrcnn_class_loss: 0.0625 - val_mrcnn_bbox_loss: 0.4573 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 4/6
200/200 [==============================] - 103s 517ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.0749 - rpn_class_loss: 0.0226 - rpn_bbox_loss: 0.5285 - mrcnn_class_loss: 0.0638 - mrcnn_bbox_loss: 0.4600 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.3396 - val_rpn_class_loss: 0.0244 - val_rpn_bbox_loss: 0.8082 - val_mrcnn_class_loss: 0.0378 - val_mrcnn_bbox_loss: 0.4692 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 5/6
200/200 [==============================] - 108s 543ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.9621 - rpn_class_loss: 0.0221 - rpn_bbox_loss: 0.5200 - mrcnn_class_loss: 0.0350 - mrcnn_bbox_loss: 0.3850 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.0859 - val_rpn_class_loss: 0.0234 - val_rpn_bbox_loss: 0.6024 - val_mrcnn_class_loss: 0.0394 - val_mrcnn_bbox_loss: 0.4207 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 6/6
200/200 [==============================] - 103s 514ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.9485 - rpn_class_loss: 0.0221 - rpn_bbox_loss: 0.5123 - mrcnn_class_loss: 0.0356 - mrcnn_bbox_loss: 0.3785 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.9917 - val_rpn_class_loss: 0.0211 - val_rpn_bbox_loss: 0.6103 - val_mrcnn_class_loss: 0.0220 - val_mrcnn_bbox_loss: 0.3383 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00

add_model_results('MR_8A')
add_best_epoch_to_list('MR_8A', 'loss', 'min')
save_run_state(RUN_ID, 'MR_8A')

Iteration 8.b) Train all layers with reduced LR 0.0002¶

STATIC_ID='MR_8B'
load_prior_run_state(RUN_ID, STATIC_ID)

class MaskConfig_8b(MaskRCnnConfig):
  IMAGES_PER_GPU=8
  OPTIMIZER='Lamb'
  LEARNING_RATE=0.0002
  STEPS_PER_EPOCH=200
  VALIDATION_STEPS=50
  LOSS_WEIGHTS={'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 0., 'dice_coeff': 0.}
  LAYERS='all'

# Config time settings - Cannot be changed at runtime.
mask_config = MaskConfig_8b()
ID=get_iteration_id(STATIC_ID)
optimizer = LAMB(learning_rate = mask_config.LEARNING_RATE)

Iteration Id: MR_8B_20210612T1428

model = get_nn_model(STATIC_ID, mask_config, optimizer)
print('Log directory:', model.log_dir)
print('Checkpoint directory:', model.checkpoint_dir)

Selecting layers to train
Log directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/logs/maskrcnn/MR_8B_20210612T1428
Checkpoint directory: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_8B_20210612T1428

weights_path = get_checkpoint_weight_path('MR_8A', 6)
model=reset_weights(ID, model, weights_path)
model_list[iteration_list[STATIC_ID]]= model

EPOCHS=10
hist=model.train(dataset_train,
            dataset_val,
            epochs=EPOCHS,
            augmentation=None) 
add_history_to_list(STATIC_ID, model.keras_model.history.history)

Checkpoint Path: /content/drive/MyDrive/Colab/CapstoneProject/dataset/data/checkpoint/maskrcnn/MR_8B_20210612T1428/maskrcnn_{epoch:04d}.h5
Model: "mask_rcnn"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_image (InputLayer)        [(None, None, None,  0                                            
__________________________________________________________________________________________________
zero_padding2d (ZeroPadding2D)  (None, None, None, 3 0           input_image[0][0]                
__________________________________________________________________________________________________
conv1 (Conv2D)                  (None, None, None, 6 9472        zero_padding2d[0][0]             
__________________________________________________________________________________________________
bn_conv1 (BatchNorm)            (None, None, None, 6 256         conv1[0][0]                      
__________________________________________________________________________________________________
activation (Activation)         (None, None, None, 6 0           bn_conv1[0][0]                   
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D)    (None, None, None, 6 0           activation[0][0]                 
__________________________________________________________________________________________________
res2a_branch2a (Conv2D)         (None, None, None, 6 4160        max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2a (BatchNorm)       (None, None, None, 6 256         res2a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_1 (Activation)       (None, None, None, 6 0           bn2a_branch2a[0][0]              
__________________________________________________________________________________________________
res2a_branch2b (Conv2D)         (None, None, None, 6 36928       activation_1[0][0]               
__________________________________________________________________________________________________
bn2a_branch2b (BatchNorm)       (None, None, None, 6 256         res2a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_2 (Activation)       (None, None, None, 6 0           bn2a_branch2b[0][0]              
__________________________________________________________________________________________________
res2a_branch2c (Conv2D)         (None, None, None, 2 16640       activation_2[0][0]               
__________________________________________________________________________________________________
res2a_branch1 (Conv2D)          (None, None, None, 2 16640       max_pooling2d[0][0]              
__________________________________________________________________________________________________
bn2a_branch2c (BatchNorm)       (None, None, None, 2 1024        res2a_branch2c[0][0]             
__________________________________________________________________________________________________
bn2a_branch1 (BatchNorm)        (None, None, None, 2 1024        res2a_branch1[0][0]              
__________________________________________________________________________________________________
add (Add)                       (None, None, None, 2 0           bn2a_branch2c[0][0]              
                                                                 bn2a_branch1[0][0]               
__________________________________________________________________________________________________
res2a_out (Activation)          (None, None, None, 2 0           add[0][0]                        
__________________________________________________________________________________________________
res2b_branch2a (Conv2D)         (None, None, None, 6 16448       res2a_out[0][0]                  
__________________________________________________________________________________________________
bn2b_branch2a (BatchNorm)       (None, None, None, 6 256         res2b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_3 (Activation)       (None, None, None, 6 0           bn2b_branch2a[0][0]              
__________________________________________________________________________________________________
res2b_branch2b (Conv2D)         (None, None, None, 6 36928       activation_3[0][0]               
__________________________________________________________________________________________________
bn2b_branch2b (BatchNorm)       (None, None, None, 6 256         res2b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_4 (Activation)       (None, None, None, 6 0           bn2b_branch2b[0][0]              
__________________________________________________________________________________________________
res2b_branch2c (Conv2D)         (None, None, None, 2 16640       activation_4[0][0]               
__________________________________________________________________________________________________
bn2b_branch2c (BatchNorm)       (None, None, None, 2 1024        res2b_branch2c[0][0]             
__________________________________________________________________________________________________
add_1 (Add)                     (None, None, None, 2 0           bn2b_branch2c[0][0]              
                                                                 res2a_out[0][0]                  
__________________________________________________________________________________________________
res2b_out (Activation)          (None, None, None, 2 0           add_1[0][0]                      
__________________________________________________________________________________________________
res2c_branch2a (Conv2D)         (None, None, None, 6 16448       res2b_out[0][0]                  
__________________________________________________________________________________________________
bn2c_branch2a (BatchNorm)       (None, None, None, 6 256         res2c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_5 (Activation)       (None, None, None, 6 0           bn2c_branch2a[0][0]              
__________________________________________________________________________________________________
res2c_branch2b (Conv2D)         (None, None, None, 6 36928       activation_5[0][0]               
__________________________________________________________________________________________________
bn2c_branch2b (BatchNorm)       (None, None, None, 6 256         res2c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_6 (Activation)       (None, None, None, 6 0           bn2c_branch2b[0][0]              
__________________________________________________________________________________________________
res2c_branch2c (Conv2D)         (None, None, None, 2 16640       activation_6[0][0]               
__________________________________________________________________________________________________
bn2c_branch2c (BatchNorm)       (None, None, None, 2 1024        res2c_branch2c[0][0]             
__________________________________________________________________________________________________
add_2 (Add)                     (None, None, None, 2 0           bn2c_branch2c[0][0]              
                                                                 res2b_out[0][0]                  
__________________________________________________________________________________________________
res2c_out (Activation)          (None, None, None, 2 0           add_2[0][0]                      
__________________________________________________________________________________________________
res3a_branch2a (Conv2D)         (None, None, None, 1 32896       res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2a (BatchNorm)       (None, None, None, 1 512         res3a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_7 (Activation)       (None, None, None, 1 0           bn3a_branch2a[0][0]              
__________________________________________________________________________________________________
res3a_branch2b (Conv2D)         (None, None, None, 1 147584      activation_7[0][0]               
__________________________________________________________________________________________________
bn3a_branch2b (BatchNorm)       (None, None, None, 1 512         res3a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_8 (Activation)       (None, None, None, 1 0           bn3a_branch2b[0][0]              
__________________________________________________________________________________________________
res3a_branch2c (Conv2D)         (None, None, None, 5 66048       activation_8[0][0]               
__________________________________________________________________________________________________
res3a_branch1 (Conv2D)          (None, None, None, 5 131584      res2c_out[0][0]                  
__________________________________________________________________________________________________
bn3a_branch2c (BatchNorm)       (None, None, None, 5 2048        res3a_branch2c[0][0]             
__________________________________________________________________________________________________
bn3a_branch1 (BatchNorm)        (None, None, None, 5 2048        res3a_branch1[0][0]              
__________________________________________________________________________________________________
add_3 (Add)                     (None, None, None, 5 0           bn3a_branch2c[0][0]              
                                                                 bn3a_branch1[0][0]               
__________________________________________________________________________________________________
res3a_out (Activation)          (None, None, None, 5 0           add_3[0][0]                      
__________________________________________________________________________________________________
res3b_branch2a (Conv2D)         (None, None, None, 1 65664       res3a_out[0][0]                  
__________________________________________________________________________________________________
bn3b_branch2a (BatchNorm)       (None, None, None, 1 512         res3b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_9 (Activation)       (None, None, None, 1 0           bn3b_branch2a[0][0]              
__________________________________________________________________________________________________
res3b_branch2b (Conv2D)         (None, None, None, 1 147584      activation_9[0][0]               
__________________________________________________________________________________________________
bn3b_branch2b (BatchNorm)       (None, None, None, 1 512         res3b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_10 (Activation)      (None, None, None, 1 0           bn3b_branch2b[0][0]              
__________________________________________________________________________________________________
res3b_branch2c (Conv2D)         (None, None, None, 5 66048       activation_10[0][0]              
__________________________________________________________________________________________________
bn3b_branch2c (BatchNorm)       (None, None, None, 5 2048        res3b_branch2c[0][0]             
__________________________________________________________________________________________________
add_4 (Add)                     (None, None, None, 5 0           bn3b_branch2c[0][0]              
                                                                 res3a_out[0][0]                  
__________________________________________________________________________________________________
res3b_out (Activation)          (None, None, None, 5 0           add_4[0][0]                      
__________________________________________________________________________________________________
res3c_branch2a (Conv2D)         (None, None, None, 1 65664       res3b_out[0][0]                  
__________________________________________________________________________________________________
bn3c_branch2a (BatchNorm)       (None, None, None, 1 512         res3c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_11 (Activation)      (None, None, None, 1 0           bn3c_branch2a[0][0]              
__________________________________________________________________________________________________
res3c_branch2b (Conv2D)         (None, None, None, 1 147584      activation_11[0][0]              
__________________________________________________________________________________________________
bn3c_branch2b (BatchNorm)       (None, None, None, 1 512         res3c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_12 (Activation)      (None, None, None, 1 0           bn3c_branch2b[0][0]              
__________________________________________________________________________________________________
res3c_branch2c (Conv2D)         (None, None, None, 5 66048       activation_12[0][0]              
__________________________________________________________________________________________________
bn3c_branch2c (BatchNorm)       (None, None, None, 5 2048        res3c_branch2c[0][0]             
__________________________________________________________________________________________________
add_5 (Add)                     (None, None, None, 5 0           bn3c_branch2c[0][0]              
                                                                 res3b_out[0][0]                  
__________________________________________________________________________________________________
res3c_out (Activation)          (None, None, None, 5 0           add_5[0][0]                      
__________________________________________________________________________________________________
res3d_branch2a (Conv2D)         (None, None, None, 1 65664       res3c_out[0][0]                  
__________________________________________________________________________________________________
bn3d_branch2a (BatchNorm)       (None, None, None, 1 512         res3d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_13 (Activation)      (None, None, None, 1 0           bn3d_branch2a[0][0]              
__________________________________________________________________________________________________
res3d_branch2b (Conv2D)         (None, None, None, 1 147584      activation_13[0][0]              
__________________________________________________________________________________________________
bn3d_branch2b (BatchNorm)       (None, None, None, 1 512         res3d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_14 (Activation)      (None, None, None, 1 0           bn3d_branch2b[0][0]              
__________________________________________________________________________________________________
res3d_branch2c (Conv2D)         (None, None, None, 5 66048       activation_14[0][0]              
__________________________________________________________________________________________________
bn3d_branch2c (BatchNorm)       (None, None, None, 5 2048        res3d_branch2c[0][0]             
__________________________________________________________________________________________________
add_6 (Add)                     (None, None, None, 5 0           bn3d_branch2c[0][0]              
                                                                 res3c_out[0][0]                  
__________________________________________________________________________________________________
res3d_out (Activation)          (None, None, None, 5 0           add_6[0][0]                      
__________________________________________________________________________________________________
res4a_branch2a (Conv2D)         (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2a (BatchNorm)       (None, None, None, 2 1024        res4a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_15 (Activation)      (None, None, None, 2 0           bn4a_branch2a[0][0]              
__________________________________________________________________________________________________
res4a_branch2b (Conv2D)         (None, None, None, 2 590080      activation_15[0][0]              
__________________________________________________________________________________________________
bn4a_branch2b (BatchNorm)       (None, None, None, 2 1024        res4a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_16 (Activation)      (None, None, None, 2 0           bn4a_branch2b[0][0]              
__________________________________________________________________________________________________
res4a_branch2c (Conv2D)         (None, None, None, 1 263168      activation_16[0][0]              
__________________________________________________________________________________________________
res4a_branch1 (Conv2D)          (None, None, None, 1 525312      res3d_out[0][0]                  
__________________________________________________________________________________________________
bn4a_branch2c (BatchNorm)       (None, None, None, 1 4096        res4a_branch2c[0][0]             
__________________________________________________________________________________________________
bn4a_branch1 (BatchNorm)        (None, None, None, 1 4096        res4a_branch1[0][0]              
__________________________________________________________________________________________________
add_7 (Add)                     (None, None, None, 1 0           bn4a_branch2c[0][0]              
                                                                 bn4a_branch1[0][0]               
__________________________________________________________________________________________________
res4a_out (Activation)          (None, None, None, 1 0           add_7[0][0]                      
__________________________________________________________________________________________________
res4b_branch2a (Conv2D)         (None, None, None, 2 262400      res4a_out[0][0]                  
__________________________________________________________________________________________________
bn4b_branch2a (BatchNorm)       (None, None, None, 2 1024        res4b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_17 (Activation)      (None, None, None, 2 0           bn4b_branch2a[0][0]              
__________________________________________________________________________________________________
res4b_branch2b (Conv2D)         (None, None, None, 2 590080      activation_17[0][0]              
__________________________________________________________________________________________________
bn4b_branch2b (BatchNorm)       (None, None, None, 2 1024        res4b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_18 (Activation)      (None, None, None, 2 0           bn4b_branch2b[0][0]              
__________________________________________________________________________________________________
res4b_branch2c (Conv2D)         (None, None, None, 1 263168      activation_18[0][0]              
__________________________________________________________________________________________________
bn4b_branch2c (BatchNorm)       (None, None, None, 1 4096        res4b_branch2c[0][0]             
__________________________________________________________________________________________________
add_8 (Add)                     (None, None, None, 1 0           bn4b_branch2c[0][0]              
                                                                 res4a_out[0][0]                  
__________________________________________________________________________________________________
res4b_out (Activation)          (None, None, None, 1 0           add_8[0][0]                      
__________________________________________________________________________________________________
res4c_branch2a (Conv2D)         (None, None, None, 2 262400      res4b_out[0][0]                  
__________________________________________________________________________________________________
bn4c_branch2a (BatchNorm)       (None, None, None, 2 1024        res4c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_19 (Activation)      (None, None, None, 2 0           bn4c_branch2a[0][0]              
__________________________________________________________________________________________________
res4c_branch2b (Conv2D)         (None, None, None, 2 590080      activation_19[0][0]              
__________________________________________________________________________________________________
bn4c_branch2b (BatchNorm)       (None, None, None, 2 1024        res4c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_20 (Activation)      (None, None, None, 2 0           bn4c_branch2b[0][0]              
__________________________________________________________________________________________________
res4c_branch2c (Conv2D)         (None, None, None, 1 263168      activation_20[0][0]              
__________________________________________________________________________________________________
bn4c_branch2c (BatchNorm)       (None, None, None, 1 4096        res4c_branch2c[0][0]             
__________________________________________________________________________________________________
add_9 (Add)                     (None, None, None, 1 0           bn4c_branch2c[0][0]              
                                                                 res4b_out[0][0]                  
__________________________________________________________________________________________________
res4c_out (Activation)          (None, None, None, 1 0           add_9[0][0]                      
__________________________________________________________________________________________________
res4d_branch2a (Conv2D)         (None, None, None, 2 262400      res4c_out[0][0]                  
__________________________________________________________________________________________________
bn4d_branch2a (BatchNorm)       (None, None, None, 2 1024        res4d_branch2a[0][0]             
__________________________________________________________________________________________________
activation_21 (Activation)      (None, None, None, 2 0           bn4d_branch2a[0][0]              
__________________________________________________________________________________________________
res4d_branch2b (Conv2D)         (None, None, None, 2 590080      activation_21[0][0]              
__________________________________________________________________________________________________
bn4d_branch2b (BatchNorm)       (None, None, None, 2 1024        res4d_branch2b[0][0]             
__________________________________________________________________________________________________
activation_22 (Activation)      (None, None, None, 2 0           bn4d_branch2b[0][0]              
__________________________________________________________________________________________________
res4d_branch2c (Conv2D)         (None, None, None, 1 263168      activation_22[0][0]              
__________________________________________________________________________________________________
bn4d_branch2c (BatchNorm)       (None, None, None, 1 4096        res4d_branch2c[0][0]             
__________________________________________________________________________________________________
add_10 (Add)                    (None, None, None, 1 0           bn4d_branch2c[0][0]              
                                                                 res4c_out[0][0]                  
__________________________________________________________________________________________________
res4d_out (Activation)          (None, None, None, 1 0           add_10[0][0]                     
__________________________________________________________________________________________________
res4e_branch2a (Conv2D)         (None, None, None, 2 262400      res4d_out[0][0]                  
__________________________________________________________________________________________________
bn4e_branch2a (BatchNorm)       (None, None, None, 2 1024        res4e_branch2a[0][0]             
__________________________________________________________________________________________________
activation_23 (Activation)      (None, None, None, 2 0           bn4e_branch2a[0][0]              
__________________________________________________________________________________________________
res4e_branch2b (Conv2D)         (None, None, None, 2 590080      activation_23[0][0]              
__________________________________________________________________________________________________
bn4e_branch2b (BatchNorm)       (None, None, None, 2 1024        res4e_branch2b[0][0]             
__________________________________________________________________________________________________
activation_24 (Activation)      (None, None, None, 2 0           bn4e_branch2b[0][0]              
__________________________________________________________________________________________________
res4e_branch2c (Conv2D)         (None, None, None, 1 263168      activation_24[0][0]              
__________________________________________________________________________________________________
bn4e_branch2c (BatchNorm)       (None, None, None, 1 4096        res4e_branch2c[0][0]             
__________________________________________________________________________________________________
add_11 (Add)                    (None, None, None, 1 0           bn4e_branch2c[0][0]              
                                                                 res4d_out[0][0]                  
__________________________________________________________________________________________________
res4e_out (Activation)          (None, None, None, 1 0           add_11[0][0]                     
__________________________________________________________________________________________________
res4f_branch2a (Conv2D)         (None, None, None, 2 262400      res4e_out[0][0]                  
__________________________________________________________________________________________________
bn4f_branch2a (BatchNorm)       (None, None, None, 2 1024        res4f_branch2a[0][0]             
__________________________________________________________________________________________________
activation_25 (Activation)      (None, None, None, 2 0           bn4f_branch2a[0][0]              
__________________________________________________________________________________________________
res4f_branch2b (Conv2D)         (None, None, None, 2 590080      activation_25[0][0]              
__________________________________________________________________________________________________
bn4f_branch2b (BatchNorm)       (None, None, None, 2 1024        res4f_branch2b[0][0]             
__________________________________________________________________________________________________
activation_26 (Activation)      (None, None, None, 2 0           bn4f_branch2b[0][0]              
__________________________________________________________________________________________________
res4f_branch2c (Conv2D)         (None, None, None, 1 263168      activation_26[0][0]              
__________________________________________________________________________________________________
bn4f_branch2c (BatchNorm)       (None, None, None, 1 4096        res4f_branch2c[0][0]             
__________________________________________________________________________________________________
add_12 (Add)                    (None, None, None, 1 0           bn4f_branch2c[0][0]              
                                                                 res4e_out[0][0]                  
__________________________________________________________________________________________________
res4f_out (Activation)          (None, None, None, 1 0           add_12[0][0]                     
__________________________________________________________________________________________________
res5a_branch2a (Conv2D)         (None, None, None, 5 524800      res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2a (BatchNorm)       (None, None, None, 5 2048        res5a_branch2a[0][0]             
__________________________________________________________________________________________________
activation_27 (Activation)      (None, None, None, 5 0           bn5a_branch2a[0][0]              
__________________________________________________________________________________________________
res5a_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_27[0][0]              
__________________________________________________________________________________________________
bn5a_branch2b (BatchNorm)       (None, None, None, 5 2048        res5a_branch2b[0][0]             
__________________________________________________________________________________________________
activation_28 (Activation)      (None, None, None, 5 0           bn5a_branch2b[0][0]              
__________________________________________________________________________________________________
res5a_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_28[0][0]              
__________________________________________________________________________________________________
res5a_branch1 (Conv2D)          (None, None, None, 2 2099200     res4f_out[0][0]                  
__________________________________________________________________________________________________
bn5a_branch2c (BatchNorm)       (None, None, None, 2 8192        res5a_branch2c[0][0]             
__________________________________________________________________________________________________
bn5a_branch1 (BatchNorm)        (None, None, None, 2 8192        res5a_branch1[0][0]              
__________________________________________________________________________________________________
add_13 (Add)                    (None, None, None, 2 0           bn5a_branch2c[0][0]              
                                                                 bn5a_branch1[0][0]               
__________________________________________________________________________________________________
res5a_out (Activation)          (None, None, None, 2 0           add_13[0][0]                     
__________________________________________________________________________________________________
res5b_branch2a (Conv2D)         (None, None, None, 5 1049088     res5a_out[0][0]                  
__________________________________________________________________________________________________
bn5b_branch2a (BatchNorm)       (None, None, None, 5 2048        res5b_branch2a[0][0]             
__________________________________________________________________________________________________
activation_29 (Activation)      (None, None, None, 5 0           bn5b_branch2a[0][0]              
__________________________________________________________________________________________________
res5b_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_29[0][0]              
__________________________________________________________________________________________________
bn5b_branch2b (BatchNorm)       (None, None, None, 5 2048        res5b_branch2b[0][0]             
__________________________________________________________________________________________________
activation_30 (Activation)      (None, None, None, 5 0           bn5b_branch2b[0][0]              
__________________________________________________________________________________________________
res5b_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_30[0][0]              
__________________________________________________________________________________________________
bn5b_branch2c (BatchNorm)       (None, None, None, 2 8192        res5b_branch2c[0][0]             
__________________________________________________________________________________________________
add_14 (Add)                    (None, None, None, 2 0           bn5b_branch2c[0][0]              
                                                                 res5a_out[0][0]                  
__________________________________________________________________________________________________
res5b_out (Activation)          (None, None, None, 2 0           add_14[0][0]                     
__________________________________________________________________________________________________
res5c_branch2a (Conv2D)         (None, None, None, 5 1049088     res5b_out[0][0]                  
__________________________________________________________________________________________________
bn5c_branch2a (BatchNorm)       (None, None, None, 5 2048        res5c_branch2a[0][0]             
__________________________________________________________________________________________________
activation_31 (Activation)      (None, None, None, 5 0           bn5c_branch2a[0][0]              
__________________________________________________________________________________________________
res5c_branch2b (Conv2D)         (None, None, None, 5 2359808     activation_31[0][0]              
__________________________________________________________________________________________________
bn5c_branch2b (BatchNorm)       (None, None, None, 5 2048        res5c_branch2b[0][0]             
__________________________________________________________________________________________________
activation_32 (Activation)      (None, None, None, 5 0           bn5c_branch2b[0][0]              
__________________________________________________________________________________________________
res5c_branch2c (Conv2D)         (None, None, None, 2 1050624     activation_32[0][0]              
__________________________________________________________________________________________________
bn5c_branch2c (BatchNorm)       (None, None, None, 2 8192        res5c_branch2c[0][0]             
__________________________________________________________________________________________________
add_15 (Add)                    (None, None, None, 2 0           bn5c_branch2c[0][0]              
                                                                 res5b_out[0][0]                  
__________________________________________________________________________________________________
res5c_out (Activation)          (None, None, None, 2 0           add_15[0][0]                     
__________________________________________________________________________________________________
fpn_c5p5 (Conv2D)               (None, None, None, 2 524544      res5c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p5upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_c4p4 (Conv2D)               (None, None, None, 2 262400      res4f_out[0][0]                  
__________________________________________________________________________________________________
fpn_p4add (Add)                 (None, None, None, 2 0           fpn_p5upsampled[0][0]            
                                                                 fpn_c4p4[0][0]                   
__________________________________________________________________________________________________
fpn_p4upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_c3p3 (Conv2D)               (None, None, None, 2 131328      res3d_out[0][0]                  
__________________________________________________________________________________________________
fpn_p3add (Add)                 (None, None, None, 2 0           fpn_p4upsampled[0][0]            
                                                                 fpn_c3p3[0][0]                   
__________________________________________________________________________________________________
fpn_p3upsampled (UpSampling2D)  (None, None, None, 2 0           fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_c2p2 (Conv2D)               (None, None, None, 2 65792       res2c_out[0][0]                  
__________________________________________________________________________________________________
fpn_p2add (Add)                 (None, None, None, 2 0           fpn_p3upsampled[0][0]            
                                                                 fpn_c2p2[0][0]                   
__________________________________________________________________________________________________
fpn_p5 (Conv2D)                 (None, None, None, 2 590080      fpn_c5p5[0][0]                   
__________________________________________________________________________________________________
fpn_p2 (Conv2D)                 (None, None, None, 2 590080      fpn_p2add[0][0]                  
__________________________________________________________________________________________________
fpn_p3 (Conv2D)                 (None, None, None, 2 590080      fpn_p3add[0][0]                  
__________________________________________________________________________________________________
fpn_p4 (Conv2D)                 (None, None, None, 2 590080      fpn_p4add[0][0]                  
__________________________________________________________________________________________________
fpn_p6 (MaxPooling2D)           (None, None, None, 2 0           fpn_p5[0][0]                     
__________________________________________________________________________________________________
rpn_model (Functional)          [(None, None, 2), (N 1189394     fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
                                                                 fpn_p6[0][0]                     
__________________________________________________________________________________________________
rpn_class (Concatenate)         (None, None, 2)      0           rpn_model[0][1]                  
                                                                 rpn_model[1][1]                  
                                                                 rpn_model[2][1]                  
                                                                 rpn_model[3][1]                  
                                                                 rpn_model[4][1]                  
__________________________________________________________________________________________________
rpn_bbox (Concatenate)          (None, None, 4)      0           rpn_model[0][2]                  
                                                                 rpn_model[1][2]                  
                                                                 rpn_model[2][2]                  
                                                                 rpn_model[3][2]                  
                                                                 rpn_model[4][2]                  
__________________________________________________________________________________________________
anchors (ConstLayer)            (8, 16320, 4)        522240      input_image[0][0]                
__________________________________________________________________________________________________
input_gt_boxes (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
ROI (ProposalLayer)             (8, 2000, 4)         0           rpn_class[0][0]                  
                                                                 rpn_bbox[0][0]                   
                                                                 anchors[0][0]                    
__________________________________________________________________________________________________
input_gt_class_ids (InputLayer) [(None, None)]       0                                            
__________________________________________________________________________________________________
lambda (Lambda)                 (None, None, 4)      0           input_gt_boxes[0][0]             
__________________________________________________________________________________________________
input_gt_masks (InputLayer)     [(None, 56, 56, None 0                                            
__________________________________________________________________________________________________
proposal_targets (DetectionTarg [(8, None, 4), (8, N 0           ROI[0][0]                        
                                                                 input_gt_class_ids[0][0]         
                                                                 lambda[0][0]                     
                                                                 input_gt_masks[0][0]             
__________________________________________________________________________________________________
input_image_meta (InputLayer)   [(None, 14)]         0                                            
__________________________________________________________________________________________________
roi_align_mask (PyramidROIAlign (8, None, 14, 14, 25 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_conv1 (TimeDistribut (8, None, 14, 14, 25 590080      roi_align_mask[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_bn1 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv1[0][0]           
__________________________________________________________________________________________________
activation_36 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn1[0][0]             
__________________________________________________________________________________________________
mrcnn_mask_conv2 (TimeDistribut (8, None, 14, 14, 25 590080      activation_36[0][0]              
__________________________________________________________________________________________________
roi_align_classifier (PyramidRO (8, None, 7, 7, 256) 0           proposal_targets[0][0]           
                                                                 input_image_meta[0][0]           
                                                                 fpn_p2[0][0]                     
                                                                 fpn_p3[0][0]                     
                                                                 fpn_p4[0][0]                     
                                                                 fpn_p5[0][0]                     
__________________________________________________________________________________________________
mrcnn_mask_bn2 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv2[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv1 (TimeDistribu (8, None, 1, 1, 1024 12846080    roi_align_classifier[0][0]       
__________________________________________________________________________________________________
activation_37 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn2[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn1 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv1[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv3 (TimeDistribut (8, None, 14, 14, 25 590080      activation_37[0][0]              
__________________________________________________________________________________________________
activation_33 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn1[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn3 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv3[0][0]           
__________________________________________________________________________________________________
mrcnn_class_conv2 (TimeDistribu (8, None, 1, 1, 1024 1049600     activation_33[0][0]              
__________________________________________________________________________________________________
activation_38 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn3[0][0]             
__________________________________________________________________________________________________
mrcnn_class_bn2 (TimeDistribute (8, None, 1, 1, 1024 4096        mrcnn_class_conv2[0][0]          
__________________________________________________________________________________________________
mrcnn_mask_conv4 (TimeDistribut (8, None, 14, 14, 25 590080      activation_38[0][0]              
__________________________________________________________________________________________________
activation_34 (Activation)      (8, None, 1, 1, 1024 0           mrcnn_class_bn2[0][0]            
__________________________________________________________________________________________________
mrcnn_mask_bn4 (TimeDistributed (8, None, 14, 14, 25 1024        mrcnn_mask_conv4[0][0]           
__________________________________________________________________________________________________
pool_squeeze (Lambda)           (8, None, 1024)      0           activation_34[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (8, None, 14, 14, 25 0           mrcnn_mask_bn4[0][0]             
__________________________________________________________________________________________________
mrcnn_bbox_fc (TimeDistributed) (8, None, 8)         8200        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_mask_deconv (TimeDistribu (8, None, 28, 28, 25 262400      activation_39[0][0]              
__________________________________________________________________________________________________
rpn_class_logits (Concatenate)  (None, None, 2)      0           rpn_model[0][0]                  
                                                                 rpn_model[1][0]                  
                                                                 rpn_model[2][0]                  
                                                                 rpn_model[3][0]                  
                                                                 rpn_model[4][0]                  
__________________________________________________________________________________________________
mrcnn_class_logits (TimeDistrib (8, None, 2)         2050        pool_squeeze[0][0]               
__________________________________________________________________________________________________
mrcnn_bbox (Reshape)            (8, None, 2, 4)      0           mrcnn_bbox_fc[0][0]              
__________________________________________________________________________________________________
mrcnn_mask (TimeDistributed)    (8, None, 28, 28, 2) 514         mrcnn_mask_deconv[0][0]          
__________________________________________________________________________________________________
input_rpn_match (InputLayer)    [(None, None, 1)]    0                                            
__________________________________________________________________________________________________
input_rpn_bbox (InputLayer)     [(None, None, 4)]    0                                            
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None, 2)            0           input_image_meta[0][0]           
__________________________________________________________________________________________________
mrcnn_class (TimeDistributed)   (8, None, 2)         0           mrcnn_class_logits[0][0]         
__________________________________________________________________________________________________
output_rois (Lambda)            (8, None, 4)         0           proposal_targets[0][0]           
__________________________________________________________________________________________________
rpn_class_loss (Lambda)         ()                   0           input_rpn_match[0][0]            
                                                                 rpn_class_logits[0][0]           
__________________________________________________________________________________________________
rpn_bbox_loss (Lambda)          ()                   0           input_rpn_bbox[0][0]             
                                                                 input_rpn_match[0][0]            
                                                                 rpn_bbox[0][0]                   
__________________________________________________________________________________________________
mrcnn_class_loss (Lambda)       ()                   0           proposal_targets[0][1]           
                                                                 mrcnn_class_logits[0][0]         
                                                                 lambda_3[0][0]                   
__________________________________________________________________________________________________
mrcnn_bbox_loss (Lambda)        ()                   0           proposal_targets[0][2]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_bbox[0][0]                 
__________________________________________________________________________________________________
mrcnn_mask_loss (Lambda)        ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
dice_coeff (Lambda)             ()                   0           proposal_targets[0][3]           
                                                                 proposal_targets[0][1]           
                                                                 mrcnn_mask[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_Mean (TensorFlowOpL [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul (TensorFlowOpLa [()]                 0           tf_op_layer_Mean[0][0]           
__________________________________________________________________________________________________
add_loss (AddLoss)              ()                   0           tf_op_layer_mul[0][0]            
__________________________________________________________________________________________________
tf_op_layer_Mean_1 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_1 (TensorFlowOp [()]                 0           tf_op_layer_Mean_1[0][0]         
__________________________________________________________________________________________________
add_loss_1 (AddLoss)            ()                   0           tf_op_layer_mul_1[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_2 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_2 (TensorFlowOp [()]                 0           tf_op_layer_Mean_2[0][0]         
__________________________________________________________________________________________________
add_loss_2 (AddLoss)            ()                   0           tf_op_layer_mul_2[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_3 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_3 (TensorFlowOp [()]                 0           tf_op_layer_Mean_3[0][0]         
__________________________________________________________________________________________________
add_loss_3 (AddLoss)            ()                   0           tf_op_layer_mul_3[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_4 (TensorFlowO [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_4 (TensorFlowOp [()]                 0           tf_op_layer_Mean_4[0][0]         
__________________________________________________________________________________________________
add_loss_4 (AddLoss)            ()                   0           tf_op_layer_mul_4[0][0]          
__________________________________________________________________________________________________
tf_op_layer_Mean_5 (TensorFlowO [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_5 (TensorFlowOp [()]                 0           tf_op_layer_Mean_5[0][0]         
__________________________________________________________________________________________________
add_loss_5 (AddLoss)            ()                   0           tf_op_layer_mul_5[0][0]          
__________________________________________________________________________________________________
tf_op_layer_AddN (TensorFlowOpL [()]                 0                                            
__________________________________________________________________________________________________
add_loss_6 (AddLoss)            ()                   0           tf_op_layer_AddN[0][0]           
__________________________________________________________________________________________________
tf_op_layer_Mean_6 (TensorFlowO [()]                 0           rpn_class_loss[0][0]             
__________________________________________________________________________________________________
tf_op_layer_mul_155 (TensorFlow [()]                 0           tf_op_layer_Mean_6[0][0]         
__________________________________________________________________________________________________
add_metric (AddMetric)          ()                   0           tf_op_layer_mul_155[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_7 (TensorFlowO [()]                 0           rpn_bbox_loss[0][0]              
__________________________________________________________________________________________________
tf_op_layer_mul_156 (TensorFlow [()]                 0           tf_op_layer_Mean_7[0][0]         
__________________________________________________________________________________________________
add_metric_1 (AddMetric)        ()                   0           tf_op_layer_mul_156[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_8 (TensorFlowO [()]                 0           mrcnn_class_loss[0][0]           
__________________________________________________________________________________________________
tf_op_layer_mul_157 (TensorFlow [()]                 0           tf_op_layer_Mean_8[0][0]         
__________________________________________________________________________________________________
add_metric_2 (AddMetric)        ()                   0           tf_op_layer_mul_157[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_9 (TensorFlowO [()]                 0           mrcnn_bbox_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_158 (TensorFlow [()]                 0           tf_op_layer_Mean_9[0][0]         
__________________________________________________________________________________________________
add_metric_3 (AddMetric)        ()                   0           tf_op_layer_mul_158[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_10 (TensorFlow [()]                 0           mrcnn_mask_loss[0][0]            
__________________________________________________________________________________________________
tf_op_layer_mul_159 (TensorFlow [()]                 0           tf_op_layer_Mean_10[0][0]        
__________________________________________________________________________________________________
add_metric_4 (AddMetric)        ()                   0           tf_op_layer_mul_159[0][0]        
__________________________________________________________________________________________________
tf_op_layer_Mean_11 (TensorFlow [()]                 0           dice_coeff[0][0]                 
__________________________________________________________________________________________________
tf_op_layer_mul_160 (TensorFlow [()]                 0           tf_op_layer_Mean_11[0][0]        
__________________________________________________________________________________________________
add_metric_5 (AddMetric)        ()                   0           tf_op_layer_mul_160[0][0]        
==================================================================================================
Total params: 45,185,182
Trainable params: 45,125,918
Non-trainable params: 59,264
__________________________________________________________________________________________________
Epoch 1/10
200/200 [==============================] - 189s 702ms/step - batch: 99.5000 - size: 8.0000 - loss: 1.3574 - rpn_class_loss: 0.0208 - rpn_bbox_loss: 0.4485 - mrcnn_class_loss: 0.3498 - mrcnn_bbox_loss: 0.5384 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.0412 - val_rpn_class_loss: 0.0227 - val_rpn_bbox_loss: 0.5742 - val_mrcnn_class_loss: 0.0302 - val_mrcnn_bbox_loss: 0.4141 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 2/10
200/200 [==============================] - 88s 443ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.8807 - rpn_class_loss: 0.0197 - rpn_bbox_loss: 0.4721 - mrcnn_class_loss: 0.0266 - mrcnn_bbox_loss: 0.3623 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 1.0272 - val_rpn_class_loss: 0.0226 - val_rpn_bbox_loss: 0.6209 - val_mrcnn_class_loss: 0.0202 - val_mrcnn_bbox_loss: 0.3636 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 3/10
200/200 [==============================] - 106s 529ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.8047 - rpn_class_loss: 0.0197 - rpn_bbox_loss: 0.4607 - mrcnn_class_loss: 0.0197 - mrcnn_bbox_loss: 0.3047 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.8658 - val_rpn_class_loss: 0.0213 - val_rpn_bbox_loss: 0.5245 - val_mrcnn_class_loss: 0.0143 - val_mrcnn_bbox_loss: 0.3057 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 4/10
200/200 [==============================] - 106s 532ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.8164 - rpn_class_loss: 0.0205 - rpn_bbox_loss: 0.4742 - mrcnn_class_loss: 0.0164 - mrcnn_bbox_loss: 0.3052 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.9648 - val_rpn_class_loss: 0.0220 - val_rpn_bbox_loss: 0.6367 - val_mrcnn_class_loss: 0.0142 - val_mrcnn_bbox_loss: 0.2919 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 5/10
200/200 [==============================] - 109s 544ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.7758 - rpn_class_loss: 0.0197 - rpn_bbox_loss: 0.4548 - mrcnn_class_loss: 0.0144 - mrcnn_bbox_loss: 0.2869 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.9312 - val_rpn_class_loss: 0.0219 - val_rpn_bbox_loss: 0.6154 - val_mrcnn_class_loss: 0.0084 - val_mrcnn_bbox_loss: 0.2854 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 6/10
200/200 [==============================] - 106s 533ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.7468 - rpn_class_loss: 0.0195 - rpn_bbox_loss: 0.4586 - mrcnn_class_loss: 0.0131 - mrcnn_bbox_loss: 0.2557 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.8680 - val_rpn_class_loss: 0.0212 - val_rpn_bbox_loss: 0.6000 - val_mrcnn_class_loss: 0.0086 - val_mrcnn_bbox_loss: 0.2383 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 7/10
200/200 [==============================] - 107s 536ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.7461 - rpn_class_loss: 0.0197 - rpn_bbox_loss: 0.4512 - mrcnn_class_loss: 0.0151 - mrcnn_bbox_loss: 0.2601 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.9300 - val_rpn_class_loss: 0.0224 - val_rpn_bbox_loss: 0.6604 - val_mrcnn_class_loss: 0.0093 - val_mrcnn_bbox_loss: 0.2380 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 8/10
200/200 [==============================] - 116s 582ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.7637 - rpn_class_loss: 0.0194 - rpn_bbox_loss: 0.4638 - mrcnn_class_loss: 0.0116 - mrcnn_bbox_loss: 0.2688 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.9447 - val_rpn_class_loss: 0.0227 - val_rpn_bbox_loss: 0.5883 - val_mrcnn_class_loss: 0.0130 - val_mrcnn_bbox_loss: 0.3208 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 9/10
200/200 [==============================] - 106s 533ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.7703 - rpn_class_loss: 0.0199 - rpn_bbox_loss: 0.4461 - mrcnn_class_loss: 0.0148 - mrcnn_bbox_loss: 0.2895 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.8740 - val_rpn_class_loss: 0.0216 - val_rpn_bbox_loss: 0.5711 - val_mrcnn_class_loss: 0.0238 - val_mrcnn_bbox_loss: 0.2575 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00
Epoch 10/10
200/200 [==============================] - 107s 534ms/step - batch: 99.5000 - size: 8.0000 - loss: 0.7297 - rpn_class_loss: 0.0196 - rpn_bbox_loss: 0.4472 - mrcnn_class_loss: 0.0123 - mrcnn_bbox_loss: 0.2505 - mrcnn_mask_loss: 0.0000e+00 - dice_coeff: 0.0000e+00 - val_loss: 0.8900 - val_rpn_class_loss: 0.0211 - val_rpn_bbox_loss: 0.5921 - val_mrcnn_class_loss: 0.0087 - val_mrcnn_bbox_loss: 0.2682 - val_mrcnn_mask_loss: 0.0000e+00 - val_dice_coeff: 0.0000e+00

add_model_results('MR_8B')
add_best_epoch_to_list('MR_8B', 'loss', 'min')
save_run_state(RUN_ID, 'MR_8B')

Plotting iteration 8 training charts¶

merged_history = pd.concat([model_results[iteration_list['MR_6']], 
                  model_results[iteration_list['MR_6A']],
                  model_results[iteration_list['MR_6B']]],ignore_index=True)

plot_graph(merged_history)

Observation: Based on the above iterations - Iteration 6 has performed better. LAMB optimizer has performed better than SGD. There was a gradual reduction in loss in case of LAMB with lowest loss = 1.64. There is better convergence in iteration 6 compared to others. Hence we will progress with 6B model as the model for evaluation

5) Model Evaluation¶

1) Test utility functions¶

def get_colors_for_class_ids(class_ids):
    colors = []
    for class_id in class_ids:
        if class_id == 1:
            colors.append((.941, .204, .204))
    return colors

def compute_dice_coeff(masks1, masks2):
  # If either set of masks is empty return empty result
  if masks1.shape[-1] == 0 or masks2.shape[-1] == 0:
      return np.zeros((masks1.shape[-1], masks2.shape[-1]))
  # flatten masks and compute their areas
  masks1 = np.reshape(masks1 > .5, (-1, masks1.shape[-1])).astype(np.float32)
  masks2 = np.reshape(masks2 > .5, (-1, masks2.shape[-1])).astype(np.float32)
  area1 = np.sum(masks1, axis=0)
  area2 = np.sum(masks2, axis=0)

  # intersections and union
  intersections = np.dot(masks1.T, masks2)
  union = area1[:, None] + area2[None, :]
  dice_coeff = 2 * intersections / union
  return dice_coeff

def compute_iou(masks1, masks2):
    """Computes IoU overlaps between two sets of masks.
    masks1, masks2: [Height, Width, instances]
    """
    
    # If either set of masks is empty return empty result
    if masks1.shape[-1] == 0 or masks2.shape[-1] == 0:
        return np.zeros((masks1.shape[-1], masks2.shape[-1]))
    # flatten masks and compute their areas
    masks1 = np.reshape(masks1 > .5, (-1, masks1.shape[-1])).astype(np.float32)
    masks2 = np.reshape(masks2 > .5, (-1, masks2.shape[-1])).astype(np.float32)
    area1 = np.sum(masks1, axis=0)
    area2 = np.sum(masks2, axis=0)

    # intersections and union
    intersections = np.dot(masks1.T, masks2)
    union = area1[:, None] + area2[None, :] - intersections
    overlaps = intersections / union
    return overlaps

def compute_overlaps_boxes(boxes1, boxes2):
  def compute_iou_boxes(box, boxes, box_area, boxes_area):
    """Calculates IoU of the given box with the array of the given boxes.
    box: 1D vector [y1, x1, y2, x2]
    boxes: [boxes_count, (y1, x1, y2, x2)]
    box_area: float. the area of 'box'
    boxes_area: array of length boxes_count.

    Note: the areas are passed in rather than calculated here for
    efficiency. Calculate once in the caller to avoid duplicate work.
    """
    # Calculate intersection areas
    y1 = np.maximum(box[0], boxes[:, 0])
    y2 = np.minimum(box[2], boxes[:, 2])
    x1 = np.maximum(box[1], boxes[:, 1])
    x2 = np.minimum(box[3], boxes[:, 3])
    intersection = np.maximum(x2 - x1, 0) * np.maximum(y2 - y1, 0)
    union = box_area + boxes_area[:] - intersection[:]
    iou = intersection / union
    return iou
  
  area1 = (boxes1[:, 2] - boxes1[:, 0]) * (boxes1[:, 3] - boxes1[:, 1])
  area2 = (boxes2[:, 2] - boxes2[:, 0]) * (boxes2[:, 3] - boxes2[:, 1])

  # Compute overlaps to generate matrix [boxes1 count, boxes2 count]
  # Each cell contains the IoU value.
  overlaps = np.zeros((boxes1.shape[0], boxes2.shape[0]))
  for i in range(overlaps.shape[1]):
      box2 = boxes2[i]
      overlaps[:, i] = compute_iou_boxes(box2, boxes1, area2[i], area1)
  return overlaps

def evaluate_model(model):
  j=0
  dice_array=[]
  iou_array=[]
  for image_id in dataset_test.image_ids:
    original_image, image_meta, gt_class_id, gt_bbox, gt_mask =\
          modellib.load_image_gt(dataset_test, maskrcnn_infer_config, 
                                image_id)
    results = model.detect([original_image], verbose=0)
    r = results[0]
    gt_index = np.where(gt_class_id == 1)
    if gt_index[0].size == 0:
      dc=None
      iou=None
    else:
      if not len(r['masks']) == 0:
        dc = compute_dice_coeff(gt_mask, r['masks'])
        iou = compute_iou(gt_mask, r['masks'])
        dc = dc.flatten()
        iou = iou.flatten()
        dc = np.sum(dc[np.nonzero(dc)])/(len(np.nonzero(dc)))
        iou = np.sum(iou[np.nonzero(iou)])/(len(np.nonzero(iou)))
      else:
          dc = 0
          iou = 0
    if not dc is None:
      dice_array.append(dc)
    if not iou is None:
      iou_array.append(iou)
    j += 1
  return dice_array, iou_array

def get_infer_config(static_id, detection_cofidence=0.75, metric='loss', mtype='min'):
  best_epoch = get_best_epoch(static_id, metric, mtype)
  weights_path = get_checkpoint_weight_path(static_id, best_epoch.index[0]+1)
  infer_config = InferenceConfig()
  infer_config.DETECTION_MIN_CONFIDENCE=detection_cofidence
  infer_config.TESTING_WEIGHTS = weights_path
  return infer_config

def get_infer_model(static_id, metric='loss', mtype='min', detection_confidence=0.75):
  infer_config = get_infer_config(static_id, detection_confidence, metric, mtype)
  model=create_inference_maskmodel(static_id, infer_config)
  return model

2) Testing Evaluation¶

load_state(RUN_ID, 'MR_6B')

model = get_infer_model('MR_6B', 'val_loss', 'min', 0.85)
dice_array, iou_array = evaluate_model(model)

print("Testing performance")
print ("===============================")
print ("Dice coefficient:", str(round(np.mean(dice_array),2)))
print ("IoU:", str(round(np.mean(iou_array),2)))

Testing performance
===============================
Dice coefficient: 0.77
IoU: 0.55

model = get_infer_model('MR_8B', 'loss', 'min', 0.85)
dice_array, iou_array = evaluate_model(model)

print("Testing performance")
print ("===============================")
print ("Dice coefficient:", str(round(np.mean(dice_array),2)))
print ("IoU:", str(round(np.mean(iou_array),2)))

Testing performance
===============================
Dice coefficient: 0.41
IoU: 0.25

3) Visualize Predictions¶

def visualize_images(test_with_id, det_conf):
  model = get_infer_model(test_with_id, 'loss', 'min',det_conf)

  fig = plt.figure(figsize=(10, 30))
  for j in range(6):
    image_id = random.choice(dataset_test.image_ids)
    original_image, image_meta, gt_class_id, gt_bbox, gt_mask =\
          modellib.load_image_gt(dataset_test, maskrcnn_infer_config, 
                                image_id)
    plt.subplot(6, 2, (j*2+1))
    visualize.display_instances(original_image, gt_bbox, gt_mask, gt_class_id, 
                                  dataset_test.class_names,
                                  colors=get_colors_for_class_ids(gt_class_id), ax=fig.axes[-1])
    plt.subplot(6, 2, (j*2+2))
    results = model.detect([original_image], verbose=0)
    r = results[0]
    visualize.display_instances(original_image, r['rois'], r['masks'], r['class_ids'], 
                                  dataset_test.class_names, r['scores'], 
                                  colors=get_colors_for_class_ids(r['class_ids']), ax=fig.axes[-1])

visualize_images('MR_6B', 0.85)

/content/drive/MyDrive/Colab/CapstoneProject/dataset/checkpoint/maskrcnn/MR_6B_20210606T1239/maskrcnn_0010.h5

*** No instances to display ***

References¶

@Project Github URL

https://github.com/amenaria/Capstone-Project-aiml/tree/master

Notebook URL - https://github.com/amenaria/Capstone-Project-aiml/blob/master/Final_Report_Submission_Capstone_Project_Computer_Vision_Pneumonia_Detection_Challenge_MaskRCNN.ipynb

	patientId	x	y	width	height	Target	class	checksum
0	0004cfab-14fd-4e49-80ba-63a80b6bddd6	NaN	NaN	NaN	NaN	0	No Lung Opacity / Not Normal	78f614e5b22357018e0c50f08bddb412
1	00313ee0-9eaa-42f4-b0ab-c148ed3241cd	NaN	NaN	NaN	NaN	0	No Lung Opacity / Not Normal	ee3b52fc977cbf1e4c2210b29221e630
2	00322d4d-1c29-4943-afc9-b6754be640eb	NaN	NaN	NaN	NaN	0	No Lung Opacity / Not Normal	0cdb347053dd580952be05667c20a593
3	003d8fa0-6bf1-40ed-b54c-ac657f8495c5	NaN	NaN	NaN	NaN	0	Normal	d016ec2dd9a89189afba8183c12a9e1e
4	00436515-870c-4b36-a041-de91049b9ab4	264.0	152.0	213.0	379.0	1	Lung Opacity	12657f14d27e815bd3026c6ffdfbae3b

	patientId	x	y	width	height	Target	class	checksum
0	0004cfab-14fd-4e49-80ba-63a80b6bddd6	NaN	NaN	NaN	NaN	0	No Lung Opacity / Not Normal	78f614e5b22357018e0c50f08bddb412
1	00313ee0-9eaa-42f4-b0ab-c148ed3241cd	NaN	NaN	NaN	NaN	0	No Lung Opacity / Not Normal	ee3b52fc977cbf1e4c2210b29221e630
2	00322d4d-1c29-4943-afc9-b6754be640eb	NaN	NaN	NaN	NaN	0	No Lung Opacity / Not Normal	0cdb347053dd580952be05667c20a593
3	003d8fa0-6bf1-40ed-b54c-ac657f8495c5	NaN	NaN	NaN	NaN	0	Normal	d016ec2dd9a89189afba8183c12a9e1e
4	00436515-870c-4b36-a041-de91049b9ab4	264.0	152.0	213.0	379.0	1	Lung Opacity	12657f14d27e815bd3026c6ffdfbae3b

	Unnamed: 0	patientId	x	y	width	height	Target	class	checksum
34643	34643	xxx dc2a32fa-d49c-4c49-bcdc-3ae6c394fbc9	158.207031	463.094849	133.645325	90.395325	1	Lung Opacity	NaN
34644	34644	xxx c36d4722-8751-4593-b04e-36b5c86055db	NaN	NaN	NaN	NaN	0	Normal	NaN
34645	34645	xxx b3cb6b71-3e7b-4287-a4e1-49a443d31215	497.970703	197.712875	231.567047	521.193909	1	Lung Opacity	NaN
34646	34646	xxx b3cb6b71-3e7b-4287-a4e1-49a443d31215	135.838257	228.696991	236.973389	616.219116	1	Lung Opacity	NaN
34647	34647	xxx 18774506-f9cb-497c-898f-0bade580f342	306.430725	77.581459	242.289429	331.935944	1	Lung Opacity	NaN

	Unnamed: 0	patientId	class	Target	row_type
0	743	1d1c51b5-1cc9-4d0c-8073-5f1cfd598f76	Lung Opacity	1	orig
1	699	66c52ba5-c91d-4f0c-af96-18148fd9dd66	Lung Opacity	1	orig
2	4447	xxx 2a55f0bc-4943-4473-8f08-5c0a36fdc929	Normal	0	aug
3	382	3862449a-cc8b-40da-91a0-a1437618e65c	Lung Opacity	1	orig
4	3671	b4bd9d9f-9dee-47f0-a734-afbfb0e554cf	Normal	0	orig

	patientId	class	Target	checksum
0	b3cbe758-859f-4ace-8968-4953ca48dfc7	Lung Opacity	1	474c09e4986ecec79ae9ceb76c1b9c7d
1	9cf3c0e7-f25c-4358-b5a9-aca2e0b6f33e	Lung Opacity	1	c5a4dd680862fecf5a31952edf6f697a
2	fff0b503-72a5-446a-843d-f3d152e39053	Lung Opacity	1	2ecae1e26cacd4fed15d7d818f4958ba
3	e0cc9552-af6e-4b2e-8744-78689b0f2aaf	Lung Opacity	1	40262a87f2c8484b7ef44c6718d645da
4	af335d18-feec-4942-9831-3e25ca20e7e3	Lung Opacity	1	4eac80b41339e8233b88e61429a35532
...	...	...	...	...
345	8214be18-4b1c-4042-a3f9-c1929c516ffb	Normal	0	78f529648b421ca121968d3bf3401e61
346	9213dc17-1267-42a4-a71d-546a4d994a4a	Normal	0	6c69493495a1691fac282eb5204e8f86
347	9709cc26-3db4-4fc4-a026-2deb53cc5bda	Normal	0	75c3db6bba4f099e41e6d3f2a3689e37
348	35ce8c4b-7dbc-49ae-b141-ee593aa2180e	Normal	0	efb5fa60459d068262ab5b1b2a339cf5
349	aa8377bb-0c43-4c48-b185-bf96cc5c6537	Normal	0	31c1073ec18e725862e8003903b24df8